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SUMMARY 



The Title IV Quality Control Study represents the first comprehensive 
evaluation of quality in the Department of Education's (ED's) major 
student financial assistance programs. The purpose of the study is to 
identify, measure* and analyze the causes of inaccurate awarding (i.e., 
error) of student aid funds. Earlier studies were predominantly aimed at 
error in the Pell Grant program. This study examines error in the Pell 
Grant, Campus -Based, and Guaranteed Student Loan (GSL) programs. 

Volume 1 of the Final Report for this study is entitled Findings . 
Data and alyses presented in that volume suggest that while corrective 
actions aimed at reducing the magnitude of error have had some success, 
there is still an unacceptably high level of error in each of the Title 
IV progran^s. The delivery mechanisms for student financial aid are 
highly error prone. While short-term corrective actions will help reduce 
specific pockets of error, changes to the structure of the delivery 
process are required to make major breakthroughs. 

In this volunie, analyses conducted to recommend and evaluate 
corrective actions to reduce error are presented. There are four major 
levels of corrective actions analyzed in this volume. The first level is 
potential means for reducing the rate and magnitude of the most 
significant student and institutional item errors. The analysis of this 
first level is found in Chapter 2. Short-term corrective actions 
relating to the first level include: 
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• The use of the Federal tax form as a tool for identifying the 
erroneous reporting of certain zero values on a student's 

•plication for financial assistance. The taK return can 
indicate the likely necessity for non-zero values for such 
items as home equity, savings, assets, and certain 
non-taxable income. 

• Reduction in household size error through either changing the 
definition of household size to be equal to number of 
exemptions, requiring a completed Verification Worksheet for 
each applicant with a section relating to household size, or 
using number of exemptions versus household size as a 
verification edit. 

• Reduction in number in college error through either verifying 
all applicants who report m^re than one in post secondary 
education or requiring a comple\.».d Verification Worksheet 
with a section relating to number in college. 

• Improvement m the forms and instructions for other 
non-taxable income, household size, and number in college. 

• Clari t ication and/or alteration of definitions and procedures 
relating to the enrollment status of clock hour, non-standard 
enrollment, and sunimer term students. 

• Broader use of institution-based quality control activities 
aimed at the identification and eradicat ion of the 
disproportionate concentration of procedural and calculation 
errors at a small number of schools. 

« Improved procedures for calculating Campus-Based and GSL need 
for those students who do not receive Pell Grants, but may be 
eligible for them. 

• Improved communication by ED with institution p res i dent s and 
financial aid adminiiztrators relating to commonly occurring 
errors and the need for standardized quality control 
procedures . 

The second level cf analysis examines the likely impact of soon to be 
implemented corrective actions with a slightly broader scope. There are 
two actions required by the recent reauthorization of the Higher 
Education Act that are analyzed. These are the use of base year, as 
opposed to prospective , income data and the reiet init ion of dependency 



status . 



Analysis of the use of prospective income data and the redefinition 
of dependency status are found in Chapters 3 and 4 respectively. Key 
results are: 

• In the Campus-Based programs, a bhift from prospective to^ 
base year income for independent students will result in a 
downward shift in need. The error rate for this data item 
will drop to less than half of its current level. 

• In the Pell Grant program, fewer than 20 percent of 
recipients will be affected by the change. Generally, the 
lower income students would not be affected. 

• Expansion of self -sufficiency criteria will greatly reduce 
the rate at which current dependent students would be 
classified as independent students under the revised 
definition . 



The third level of analysis pertains to corrective actions that 
involve a change in the focus or level of Federal oversight activities. 
In Chapter 5 we examine two such corrective actions, institution-based 
quality control and post hoc application data item validation by 
institutions. The major conclusions are: 



• Institution-level quality control procedures are associated 
with lower institutional error. The results support 
continued expansion of the Institutional Quality Control 
initiative to additional institutions and the provision of 
technical assistance relating to quality control procedures 
to all schools. 

• Confirming similar analysis presented in Findings , post hoc 
validation is successful in removing targeted item errors. 
We analyzed the association between various procedures for 
conducting validation and the amount of error removed through 
validation in an attempt to identify particularly successful 
techniques . However, no significant differences were found 
among the various procedures. This indicates that improved 
targeting of applicants for validation is the remaining tool 
for increasing the efficiency of validation. 
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The fourth level of analysis is aimed at corrective actions focusing 
on long-term, structural improvement in the delivery of student aid. 
This corrective actirr' focus is the subject of a separate volume entitled 
Delivery System Quality Improvements > In that volume a multi-phased 
approach to implementing six quality improvements is presented. These 
quality improvements represent significant changes in the delivery 
process and hold out the best hope for removal of the currently high 
levels of error. One such improvement, is the /eduction in the number 
and complexity of application data items for the Campus-Based programs. 
Analysis of this possible corrective action is presented in Chapter 6 and 
indicates very positive outcomes can be achieved by significantly 
reducing the number of data elements currently required. 

The current study is not the first to examine the quality of the 
Title IV progrars. Previous studies have found error to be high in the 
programs, and they have provided the basis for corrective action 
recommendations and improvements made in the reauthorization of the 
Higher Education Act. These recommendations and corrective acti:^ns 
include : 

• Including and monitoring quality control requirements in each 
ED contract involving the delivery of student aid 

• Introducing error-prone modeling techniques for the selection 
of Pell Grant recipients for institutional validation and 
development of comprehensive edits of application data 

• Extending the validation requirements to include the 
Campus -Based and GSL programs and continued training 
opportunities for institutional f inane ial aid and fiscal 
personnel 

• Increasing the number of applicants chosen for validation 
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• Increasing the number of data items to be verified 

• Hatching application data with other Fsdsral sources of 
financial information 

• Developing management initiatives tz expand quality control 
at the institution level 

• Changing how a student's dependency status is determined 

• Developing a shortened Pell form for low income families 

• Decreasing the use of prospective income in the Pell and 
Campus-Based programs substantially 

The results of analyses of findings and corrective actions indicate 
that ED faces a critical decision in improving the quality of the Title 
IV delivery system. Error continues to be high in spite of corrective 
actions already taken. Yet th2 corrective actions ED has taken have 
nearly exhausted the options for using mechanical approaches to reducing 
error in individual data items. ED must eitner accept error rates of the 
magnitude that currently exist, including the reliance on costly 
after-the-fact inspection techniques, or accept the challenge of 
restructuring and simplifying the delivery system itself. 

Our specific recommendations and findings are summarized in Exhibit 
1, However error is defined, broadly to include all regulatory and 
procedural requirements or narrowly to include only financial liability, 
it seriously undercuts the basi^ objectives of equity and fairness. The 
remainder of this report addresses this issue in detail. 
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(UEVELS 1 9nd U) 



LONG-TERM 
(LEVELS III AND IV) 



To addrets eirort in horrvi •quity. Mvingt. assets, and certain non>taxablt inooma itams. ws 
racorrmiand using tha Fadaral tax form asa tool for identifying tha anonaous rapoding of oartain 
zaro values on a students application for f nanctal MistvKa. 

For errors in houMhold size, number in coffaga. and other non-taxable inooma wi recommend 
ki^vementstotheiaspactivaforrnaandinstfxictiuns. Wa alio raoommend ueing the Fedi^al tax 
form to indttale the presence of certain non-taxiMe items and tha nurr^ 
to flag possible erroneous houMhotd sizes (or even changing the definition of household size lo 
equal ruimbar of exemptions). 

We recomnrvend duifying and/or altering definitions ana procedures relating to the enrolment sutus 
of dock hour, non-standard enrofiment, and summer term students. This includes als;ling schools 
to error-pror<e situations and darifying aHowanoes for summer sesaiona. 

We.'scommend broader uae of inatitution-basad quality control activitiaa aimed at identifyina vid 
eradicating the disproportionate concentration of procedural and calculation errors at a tmml number 
of achods. 

V;e recommend improved procedures for calculating Campus-Baaed and QSL need for thoee students 
1^ do not receh^ PeB Grants, but may be eligible for them. espaciaVy by darifying th^ 
^ibiltty versus Pelt award 

We recommend impc9ved communication by ED with institution presidents and financial aid 
administrators relating to commonly occurnng errors and the need for standardized quality control 
procedures. 

We recommend expanding self-sufficiency criteria in the new definition of dependency status fo 
greatiy reduce the rate by which current dependent students woUd be dassifiad as independent 
students. 



Our analyses show that institutton-level quality control procedures are associated with lower institutional 
error. The results support continued expansion of the Institutional Quality Control initiative and providing 
technical assistance relating to quality conb'ol procedures to aK schods. 

Our analyses show that validation is successful in renwving errors in targeted items. We analyzed the 
assodation between various procedures for conducting validation and me amount of error removed 
through validation in m\ atternpt to identify particUarty successful technic, jes. However, no sionificant 
differences were found anwng the various procedurea. Thia leada us to recommend that the best 
chance for improved results is better selection of error-prone cpplicationa for validation. 

Our analyses show thatredudng the number and complexity of data items in the Uniform 
Methoddogy {UM) indicates that error can be reduced in the Campus-Based and QSLprograms 
without eigm/cantiy affecting the distributions of need and certification. We recommend that ED 
proceed with Oie design and imptementation of a reduced iata element needa analysis fom^tula. 

We recommend ED embark on a fong-term plan to intprove the structure of studer.t aid del'rvery 
Spedfic recommendatx>ns are found in a separate volunie. Delivery System Quality Improvements. 
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ACTION RECOMMENDATIONS AND ANALYSES 
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IMTRODUCTIOH 



The ultimate purpose of the Department of Education's (ED's) quality 
control studies is to reduce error and improve the quality of the Title 
IV programs.^ Thus, an important component in ED's quality improvement 
strategy is the analysis and implementation of corrective actions. This 
document reports on corrective actions based on findings from Stage Two 
of the Title IV Quality Control project. Two basic types of analyses are 
presented in this report: those that lead to developing corrective 
actions, and those that assess (where possible) the likely effects of 
corrective actions. In this chapter we present highlights from Findings , 
the framework for identifying and classifying corrective actions, and 
consequences that the study design places on our ability to develop and 
assess corrective actions. 

1.1 HIGHLIGHTS OF FINDINGS 

The Department of Education (ED) has designed a comprehensive quality 
program for assessing the delivery of student financial aid. This 
quality program has five far-reaching objectives that affect the 
direction and focus of its related activities. These objectives are the 
following: 



Ifor a detailed description of the Title IV programs, please see 
Appendix A to Findings « 



. TO develop procedures to define quality in che Title IV programs; 

. TO identify, measure, and evaluate deviations fro. quality (the 

occurrence of error); 
. TO determine the causes and factors affecting major errors; 
. TO identify, analyse, and implement corrective actions; and 
• TO monitor the effects of corrective actions on quality. 

AS one in a series of quality control studies designed to measure the 
quality of the student financial aid programs, the current study is part 
Of an ongoing attempt to increase awareness within ED of the consequences 
of a lacy of quality. These quality control studies have included: the 
pell Grant Quality Control contract, which measured error in the Pell 
program for the 1980-81 and 1982-83 academic years; Stage One of the 
Title IV Quality Control contract, which tested a methodology to measure 
error in the Campus-Based and GSL programs for the 1983-84 academic year; 
and the current study - Stage Two of the Title IV contract, which 
constitutes the first integrated study to measure error in the Pell, 
Can.pus-Based, and GSL programs. The current study measured error in the 
1985-86 academic year. 

In the previous volume of this study, Findinas, we presented our 
estimates of the extent and type of error in the Title IV programs. In 
Findings , we presented overall estimates of error for each of the Title 
IV programs and apportioned the error to students, institutions, and 
finally to individual student application items and institutional items. 
In addition, we tested a variety of characteristics of students and 
institutions to determine if the characteristics were associated with 
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error. This dociL-ntnt ^amines the errors presented in Findings and their 
source and determines probable causes of the errors. Based on the 
probable causes, this report will assess alternative corrective actions 
aimed at these causes. Following are highlights of the analyses 
presented ia Findings ■ 

1.1.1 Error in the Pell Grant Program 

Due to the ^latively strict rules that govern the Pell Grant 
program, payment co:isequences of errors in application or institutional 
items can be measured accurately. From these calculated payment 
consequences. Stage Two analyses produced the following findings 
concerning error in the Pell Grant program: 

o About 54 percent of 1985-86 Pell Grant recipients had errors 
which resulted in changes in award. The frequency of 
overawards was more than double that of underawards. 

• Absolute program-wide payment error (adding overawards to 
underawards) totalled $763 million or 21 percent of total 
program funds awarded. 

• N'Jt program-wide error (overawards offsetting underawards) 
totalled $407 million. 

• Nearly 32 percent of the recipients had student errors which 
resulted in $439 million in absolute program-wide payment 
error . 

• Errors in non-taxable income and reported home equity 
resulted in $75 million and $64 million, respectively, in net 
payment error. Prospective items as a group, including 
household size and number in college, contributed about $78 
million to Pell student error. 

• Thirty percent of the recipients had institutional errors. 
These errors resulted in $386 million in absolute program 
error, or 11 percent of program funds. Errors in determining 
enrollment status were the most frequent institutional errors. 
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These highlights demonstrate cne neea to xmy^^^e^^^ _c._on. 

for errors in the Pell Grant Program. 

1.1.2 Error in the Campus -Based Programs 

Error measurement in the Campus-Based programs first requires 
calculating changes to need, a measure of a family's ability to pay for 
education, which takes into account the cost of education and other 
financial aid available. This section presents the findings for need 
error and awards in excess of need. Campus-Based distributional error is 

2 

not discussed in this report. 

• Need error (both increases and decreases in need) occurred 
in 77 percent of cases, with net need error (increases 
offsetting decreases in need) totalling $504 million in need 
overstatements. Awards in excess of need (cases where best 
need fell below award) were present in 22.5 percent of the 
cases and totaled $265 million. 

• Student need error occurred in 65 percenr of the cases and 
totaled $403 million net. Institutional need error 
occurred in 32 percent of the cases and totaled $100 million 
net . 

. Errors in estimating expected income led to the largest need 
errors for individual items. Errors in expected taxable and 
non-taxable income ^-ach resulted in $114 million in net need 
«.rror. As a group cf items, prospective items resulted in 
$319 million in net nead error. 

• Institutional error most often occurred due to errors in 
factoring Pell awards in Campus-Based need. However, 
institutional errors in disbursement and initial overawards 
caused the largest dollar errors, $42.8 million and $47.2 
million, respectively, to be awarded in excess need. 



2campus-Based distributional error translates the effects of need 
errors to likely changes in awards by using institutional packaging 
constraints and parameters. 



Because these errors demonstrate a significant deviation from quality 
in the Campus-Based programs, corrective actions are needed to reduce 
these errors. 



1.1.3 Error in the GSL Program 

Error in the GSL program is measured at the point of certification of 
a loan's ) by the institution and is defined as the decrease in 
certification amount when best values are used, subject to maximum loan 
limits. (Only overcerti f ications are considered errors in the GSL 
program for purposes of this study.) The GSL estimates do not represent 
costs to the government; costs to the government are estimated by taking 
into account actual loan amounts and government costs per dollar loaned. 
Following cvre highlights of Findings on overcertif ication in the GSL 
program. 

. Approximately 20 percent of the cases had GSL certification 
error, totalling $920 million in overcertif ications of loans 
program-wide . 

• Student errors causing overcertif ications occurred in 10.6 
percent of the cases and accounted for $393 millio'. 
program-wide. Institutional errors were slightly more 
prevalent in 13.5 percent of the cases, and totaled $587 
million program-wide. 

• Institutional error was most often attributed to errors in 
determining EFC. This occurred in 6.2 percent of the cases 
and accounted for $260 million program-wide, which was also 
the largest dollar error. 

These findings indicate that G3L error was significant and that 
corrective actions are necessary to reduce the error. 
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nal Quality Control 

In addition to measuring the level of error in each of the Title IV 
programs. Findings focused on the effectiveness of two key efforts to 
improve quality in the Title IV programs. We analyzed ED mandated 
validation in the Pell program and optional institutional validation 
activities in Pell, Campus-Based, and to a limited degree, GSL. Findings 
also examined voluntary institutional quality control procedures. The 
study data concerning validation and institutional quality control 
procedures indicated the following: 

• Validation, mandatory and voluntary, occurred for 80 percent 
of the Pell Grant recipients. 

• Studants selected for validation by the Pell Processor had 
the highest rates of error on their initial applications for 
the six data items mandated for validation. 

• After validation the remaining error in these six items was 
not much different for Pell selected, institution selected, 
and non-validated students. 

• Despite not targeting well, institutions do a good job of 
removing potential error through validation for those 
students with discrepancies. 

• After validation error still remains high. Pell selected 
recipients had $161 error per student, institution selected 
recipients had $127 error per student, and non-selected 
students had $207 error per student by the time final awards 
were made. 

^ Validation in the Campus-Based programs was virtually as 
extensive as in Pell, with recipients receiving awards from 
multiple programs most liJcely to be validated. Pell 
validated Campus-Based recipients selected by the Pell 
processor had the lowest error rates. 

• Validation in the GSL program is relatively rare. 
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• Pell and Campus-Based recipients attending institutions that 
used particular typ<JS of quality control procedures 
extensively (e.g.. sampling) had a significantly lower 
institutional error rate, 

• Recipients attending institutions with little or no quality 
control procedures had the highest institutional error rates. 

Validation and institutional quality control represent brO'id 
strategies for reducing orror in the Title IV programs. These findings 
indicate that corrective actions building on these strategies should be 
examined. 

1.2 CORRECTIVE ACTIONS FRAMEWORK 

As part of its cohesive corrective actions strategy, ED has conducted 
n'jimerous quality control studies of the student financial aid programs. 
Consistently/ these studies have found quality problems to be a major 
concern for each of the programs. Because error has been pervasive at 
all levels (item-level as well as system-wide), it is necessary to 
examine many different levels of corrective actions. Four alternative 
levels include corrective actions aimed at errors in individual data 
items or individual components o£ the delivery systems, corrective 
actions for groups or classes of similar data items or components of the 
delivery systems, corrective actions based on changes in strategic 
approaches to improving quality, and corrective actions aimed at major 
problems in the programs that involve structural changes in the delivery 
system. 



Previously, #ach of these types of corrective actions has been 
determined to have merit- Therefore, the current study and investigation 
of corrective actions will take the same approach as previous studies and 
will examine corrective actions at all four levels. These levels are 
differentiated as follows: 

♦ Level I - corrective actions designed in response to 
significant errors in individual dac^> items or individual 
components of the delivery system, often as short-term 
measures ; 

♦ Level II - corrective actions oriented towards groups or 
classes of data items, or types of components of the delivery 
system; 

♦ Level III - corrective- actions that constitute a shift in the 
approach to quality; and 

♦ Level IV - corrective actions that are longer-term and 
involve major, in many cases structural, changes in the 
delivery systems or tne Title IV progran^s. 

Because there exists a range of possible corrective actions for 

errors or problems in the delivery system, it is necessary to look across 

all levels for evaluating alternatives • These four levels are 
distinguished by: 



♦ The time frame or focus to which the corrective actions are 
oriented (short-term, intermediate-term, or long-term); 

♦ The types of activities or chances required to implement the 
corrective actions (administrative decisions, regulatory 
changes, or changes in legislation); 

♦ Changes in the roles of the participants in the delivery 
system caused by the corrective actions; and 



♦ Whether or not the corrective actions would require the use 
of new technologies ♦ 



Exhibit 1-1 sunsnarizes the alternative levels of corrective actions 
and how they differ along these four parameters. In addition to 
summarizing how Level I, II. HI. and IV corrective actions differ. 
Exhibit 1-1 also shows corrective actions that are representative of tach 
of the levels. 

These four levels of corrective actions translate naturally into a 
strategy for improving the quality of the Title IV delivery system. 
Level I corrective actions will reduce error in the items towards which 
they are targeted, yet Level I corrective actions are not able to address 
the majority of error inherent in the delivery system. Much residual 
error will remain after implementing Level I corrective actions. 

Level II corrective actions will affect error in data groups within 
the delivery system. Because Level II corrective actions affect larger 
aspects of the delivery system, they will remove more error than Level I 
corrective actions. However, even after implementing Level II corrective 
actions, a significant amount of residual error Jill remain. 

Level III corrective actions are oriented towards processes within 
the delivery system. In particular. Level III corrective actions are 
targeted on reducing error by restructuring procedures, lines of control, 
and authority within the delivery system. By implementing Level III 
corrective actions most residual error will be removed. The remaining 
residual error after Level III corrective actions will be the portion 
inherent in the delivery system. 
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DIFFERENTIATINQ CHARACTERISTICS 



I 

O 



LEVEL OF 

CORRECTIVE 

ACTION 



LEVEL I 



LEVEL II 



LEVEL III 



LEVEL IV 



FOCUS 



SHORT-TERM TO 

MTKRMEOMTE- 

TtRM 



NTERMEDIATE- 
TERM 



METHOD 

REQUIRED 

FOR 

IMPLEMENTATION 



INTERME04ATE-TERM . 
TO LONG-TERM 



LO^^TERM 



AOM»itSTRATIVE 

LCCISKDN 



AOMMtSTRATIVE 
DECISION OR MINOR 
REGULATORY CHANGE 



REGULATORY CHANGE 
OR LEGISLATIVE 
CHANGE 



LEGISLATIVE CHANGE 



EXTENT OF 
CHANGE IN 
ROLES AMONG 
PARTICIPANTS 



NONE 



NONE 



MAJOR CHANGES 
LIKELY 



MAJOR CHANGES 
LKELY 



EXTENT OF NEW 
TECHNOLOGIES 
REQUIRED 



NONE 



• AUTOMATION*^ 
EXISTilGWC-. t 
PROCESSES 




SOFTWARE 
MOOmCATXM 
MINOR SOFTWARE OR 
SYSTEM REDESIGN 



* SCFfWARE 

* HAROWAf^ 
« MAX)R SYSTEM 

REDfeSIGN 

* DBMSIMPiEMEKT- 
ATlONOa 
CONVERSIOH 



EXHIBIT 1-1. DIFFERENTIATING CHARACTERISTICS OF ALTERNATIVE 
LEVEIS OF CORRECTIVE ACTIONS 
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The residual error remaining af*^^r Level III corrective actions can 
only be affected by Level IV corrective actions, which are oriented 
towards structural aspects of the delivery system. Level IV corrective 
actions will remove most or all of the error-prone components of the 
delivery system and will minimize residual error. Exhibit 1-2 summarizes 
these aspects of the corrective action framework and its implied 
strategic approach to reducing the residual error in the Title IV 
delivery system. 

These four levels of corrective actions are descriptive of the 
majority of possible corrective actions. However, these levels do not 
exhaust the universe of possible combinations of the four differentiating 
parameters and should not be inferred as the only differentiating 
characteristics. Therefore, all components for each differentiating 
characteristic as indicated in Exhibit 1-1 need not be present for all 
corrective actions at each level. For example, not all Level IV 
corrective actions will require the types of major technology changes 
indicated in Exhibit 1-1- However, given the nature of Level IV 
corrective actions, these types of technology changes are likely, 

1.2.1 Level I Corrective Actions 

Level I corrective actions may include changes in individual data 
items or individual components of the delivery system, changes to forms, 
or changes in timing. Clarifying or modifying the procedures for 
determining enrollment status is an example of a Level I corrective 
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CORRECTIVE 
ACTIOH 

LEVEL 


CORRECTIVE ACTION COMPONENTS 


TARGETS OF 
IMPLEMENTINQ CORRECTIVE ACTIONS 


LEVEL 1 


* 9^nEASCD VESICATION OF INDfmiAL 
DATA rTEMS OR INDMDUAL PnOCEDURES. 

- iUV>RGVEMENrS TO FORMS 


• REDUCED ERROR M TARGETED ITEMS 

• MNMALLKELY EFFECTIVENESS OF 
CORRECTIVE ACTIONS DUE TO NATURE OF 
DATA ITEMS AND ERROa 

• MUCH RESOUAL ERROR REKMNS 


LEVEL n 


* a^AI^SNDEFff^riONSORTREATMEriTOF 
DATA riEMS OR PROCCOMS. OR IN GROUPS OF 
DATA rTEMS OR PROCEDURES. 


• CRmCALOATArTHMSRESTRUCTUREDTO 
REDUCE ERR0R4>R0NE CHARACTERSTtCS 

• SOME RESOUAL ERROR REMOVED, DUE TO 
PROBLEMATIC DELIVERY SYSTEM ASPECTS. 


LEVEL III 


• WSTfTUTIONAlQUALnYCONTna 
PHOUcDUHcS. 

• CHANGES IN VAUOATION PROCEDURES TO 
MCREASE EFFECTIVENESS 


• THROUGH CHANGES M ROLES OR RESPONSeiHY. 
OR OTHER OaiVERY SYSTEM PROCESSES. 
MCREASED EFFECT^ENESS OF QUALTTY CONTRa 
TACTICS ! 

• ALTERATIONS MERKOR-PRONE DELIVERY 
SYSTEMS REMOVE MOST RESDUAL ERROR 


LEVEL IV 


• MAJORCHANGES TO DELIVERY SYSTEM THROUGH 
ELNylMATON OF ERROR-PRONE DATA rTEMS. 

• WTEGRATK)N OF COMPONENTS OF DELIVERY 
SYSTEMS. 


• BY SIGNnCAKaY RESTRUCTURNQ DELIVERY 
SYSTEM. REMOVE MOST OR AU OF THE 
ERROR-PRONE COMPONENTS ■ 

* RESDUAL ERROR N DELIVERY SYSTHMMMMCED. 1 



EXHIBIT 1-2. TRANSLATING ED'S CORRECTIVE ACTION FRAMEWORK 
INTO A CORRECTIVE ACTION STRATEGY 



action. This corrective action is short-term in its focus, holds as 
constant almost all parameters within the delivery systerr., and would 
require only an administrative decision to implement. In addition, the 
roles of individual parties (schools, ED, etc.) within the delivery 
system are not likely to change, nor will this corrective action require 
new technologies. We will discuss Level I corrective actions in Chapter 
2 and will focus on significant student and institutional errors 
identified in Findings . 

1.2.2 Level II Corrective Actions 

Level II corrective actions involve changes in classes of data items 
or groups of procedures in the delivery system, and are similar in many 
ways to Level I corrective actions. The major difference between 
Level II and Level I corrective actions is that Level II corrective 
actions involve a sliahtly larger scope. Changing from prospective to 
base year income items is a Level II corrective action and is 
intermediate-term in focus. This action is likely to require regulatory 
changes to implement, and tends to work within the constructs of the 
existing delivery system by accepting as constant most parameters of the 
programs. In addition, this corrective action is not likely to change 
the roles of the participants, or require major changes in technology. 
Chapter 3 will locus on Level II corrective actions and discuss the group 
of prospective income data items; and. Chapter 4 focuses on a Level II 
corrective action that looks at a new definition of dependency status and 
the group of data items that constitute the new and old definitions. 
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1*2.3 Lev«l III Cottactive Actions 

Typical Level III corrective actions would include a change in the 
focus of Federal oversight or quality assurance, or a change in the level 
at which these oversight activities are conducted. The Institutional 
Quality Control Pilot Project is an example of a Level III corrective 
action. The Pilot is intermediate to long-term in its focus and changes 
more aspects of the delivery systems than either of the two previous 
corrective action levels. By making institutions the focus of quality 
control, the Pilot will change the relationships between schools and ED. 
The Pilot involves technologies new to the current delivery system in the 
form of software for both institutions and ED. The Pilot has required a 
regulatory Change and may require a legislative change to implement 
fully. In Chapter 5 we focus on two Level III corrective actions, 
institutional quality control procedures and validation of 
student-reported data. 

1.2.4 Level IV Corrective / ;tions 

Level IV corrective actions include major changes in the formulae 
used to calculate awards in the programs or integration of components of 
the delivery systems. These corrective actions focus on long-term 
activities and require substantial changes to most aspects of the 
programs or delivery systems. Level IV corrective actions are likely to 
require legislative changes to implement, could possibly redefine the 
roles of the participants in the systems (e.g., if processing occurred at 
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simt>lifi«d formula), and are likely to require 
technologief or processing techniques that are new to the delivery system 
including new software, and perhaps changes in hardware. In Chapter 6 we 
focus on data element reduction, a Level IV corrective action that 
changes the structure of the programs by simplifying the formulae used to 
calculate need or award. 

In the past, ED's corrective action efforts have focused primarily on 
Level I and Level II corrective actions. Even with these corrective 
actions, error continues to be high. ED must decide if it is willing to 
accept the current level of error and maintain the status quo, or address 
system deficiencies through Level III and Level IV corrective actions, 
and design error out of the system. Because future reductions of error 
are most feasible through Level III and Level IV corrective actions, this 
report emphasizes these corrective actions, including analysis of 
institutional quality control procedures and simplification of the 
delivery system and need analysis formulae. 



1.3 LIMITATIONS OH ANALYSIS OF CORRECTIVE ACTIOHS 



The analyses of the corrective actions in this report have been 
designed to determine the probable causes of the errors presented in 
Findings . However, even with these additional analyses, there are some 
limitations on the extent to which they will convey underlying causal 
relationships, the extent to which corrective actions can be developed, 
and the impact simulated with the Stage Two data base. 



These limitations are a result of choosing a study design that 
generates accurate national estimates of error rather than one that 
develops and fully analyzes alternative corrective actions. The study 
design used for Stage Two was a function of its goals and constraints. 
Because Stage Two sought to maximize the precision of national error 
estimates while minimizing the costs of the study, both in terms of 
financial costs as well as the burden of collecting the data, the study 
generated a data base that reflected these goals and constraints. The 
characteristics of the data base can be summarized as follows: 

• It does not contain many student and institutional items that are 
not directly related to calculating awards or determining need. 

• Institutions' responses to questions concerning institutional 
practices are self reported rather than generated by observation 
or corroboration. 

• It contains only recipients of Title IV aid (e.g., received a Pell 
or Campus-Based award, or were certified for . GSL) and not 
applicant non-recipients. 

• It contains a large number of students overall, but only a small 
number of students per institur.ion. 

• It does not contain any operational cost or burden data. 

There are broad consequences of using a data base with these 
characteristics to develop and analyze corrective actions. First, 
because data on many student and institutional items not directly related 
to the calculation of awards were not collected, we have a limited 
capability to control for these items in our analyses and so are limited 
in our ability to determine causal relationships. In addition, we cannot 
fully simulate the effects of corrective actions because many data that 
we would need to do so are not contained in the data base. In some cases 
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(as in the need to have income data from 2 years prior to the academic 
year in Chapter 5), we can make some reasonaole assumptions concerning 
these values. In other cases, such as determining how Campus-Based 
awards would change under alternative corrective actions, we cannot make 
reasonable assumptions because the corrective actions represent a 
fundamental change to the environment in which awards would be made. 
Because we collected no cost or burden data, it is extremely difficult to 
estimate the costs of alternative corrective actions. 

Second, because data on institutional procedures were collected 
through interviews with Financial Aid Administrators, we cannot control 
for variations in interpretation or meaning across institutions. Thus, 
while two different institutions may both claim that they employ sampling 
as a quality control procedure, in actuality their sampling procedures 
may be very different. 

Third, because the data base contains only recipients and not 
applicants, we were constrained in our ability to estimate the 
possibility of new recipients being eligible under alternative corrective 
actions. This may become critical when a corrective action represents a 
fundamental change in the Title IV programs. 

Fourth, in order to maximize the precision of the national error 
estimates for a given sample size, a small n-amber of students (relative 
to the populatioi- of racipients at the institution) were sampled from 
each of a relatively large number of institutions. Consequently, no 
estimates are possible at the individual institutional level. 



However, the Stage Two data base is unique in its ability to support 
analyses of corrective actions based on changes in how student items are 
treated* Few other data bases contain recipient and family information 
to the extent that the Stage Two data base does. Therefore, wa have used 
thes) data in analyzing the effects of corrective actions relating to 
prospective income, dependency status, and a shortened need analysis 
formula. In the following paragraphs we discuss the types of analyses we 
use in each of the chapters of this report. 

In Chapter 2 we develop and assess corrective actions for errors in 
individual student and institutional data items. The conclusions 
presented are limited because corrective actions for individual items are 
constrained ' within the existing constructs of the programs. For 
corrective actions aimed at student errors, this constraint orients 
corrective actions toward validation of individual items. Developing 
corrective actionc directed at individual institutional items or 
procedures is further constrained by the sampling design employed by the 
Stage Two study. Because the study did not draw student samples that are 
statistically representative of the institutions from which they were 
drawn, we do not have the capability to fully analyze errors at the 
institutional level . 

In Chapters 3, 4, and 6, we look at changes in program definitions. 
The corrective actions in Chapter 3 examine the group of prospective 
income items and seek to assess the effects of proposed changes in how 
these items are used. Chapter 4 looks ?.t another corrective ac^ion ED 
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has i«pl««.nf d since collecting d... for this study: the likely effects 
of chan9ing th. definition used to determine dependency status. Chapter 
6 looks at reducing the number of data elements used in the Uniform 
Methodology. All three of these chapters use a series of analyses that 
describe the distribution of awards or need under the new and old 
definitions using both reported and best data. 

Finally, in Chapter 5 we look at ED's broad quality strategies of 
validation and institutional quality control aim,d at the entire delivery 
system. The analyses in Chapter 5 were structured to develop corrective 
actions within the framework of these strategies. The presentations in 
Chapter 5 use multivariate regression analysis to control for factors not 
controlled for in Findings . We are unable to state causal relationships 
in Chapter 5 because we could not control for all variables, nor could we 
control for differences in school responses given the sel f -reporting of 
many of the explanatory factors. 
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COBRECriVB ACTIOHS FOR SIGMiriCAHT STUDENT AMD IMSTITUTIOHXL EBBORS 



This chapter presents our analyses of significant individual errors 
and corresponding corrective actions. All of the corrective actions in 
this chapter are Level I corrective actions, dealing with only individual 
data items or procedures. The results are presented in two sections: 
significant student errors and significant institutional errors. The 
errors considered in this chapter were those marginal errors found to be 
significant in Chapter 5 of Findings . 

Findings described the magnitude and frequency of the significant 
errors. In general, the analyses in this chapter are designed to go 
beyond Findings , to examine in greater detail the errors that occurred 
and to suggest ways of correcting them. Following is a summary of these 
analyses . 

Corrective Actions For Significant Student Errors 

A significant percentage of misreporting of home equity, savings, 
dependent student's assets, and other non-taxable income was due to 
erroneous reporting of a zero value for these data items. Of all 
applicants, the percentage who reported a zero value incorrectly are as 
follows : 



Parent's home equity 


15,4% 


Independent student's home equity 


6.5% 


Parent's savings 


37,3% 


Independent student's savings 


28,7% 


Dependent student's assets 


31*3% 


Other non-taxable income 


21*6% 



The erroneous reporting of zero occurrs in items for which values may 
be cross-checked on the Federal tax form filed by the applicant or 
his/her parentis ). While none of the values of these items can be 
obtained directly from the tax return, the tax return can indicate 
situations where a data item exists when none was reported on the 
application* Therefore, using the Federal tax form as a source of 
information to determine when values should be verified as opposed to 
using the form as the source of verifying values should be 

investigated by ED. 

In addition, changes in application forms and clarification of 
instructions could improve the accuracy of several data items, including 
other non-taxable income, household size, and number in college. 

Corrective Actions For Significant Institutional Errors 

Enrollment status errors in the Pell program suggest that 
institutions hav difficulty calculating enrollment status correctly for 
non-standard students. Institutions seem to have trouble adjusting a 
student's enrollment status for summer sessions and when the student 
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changes enrollment st«tus . Also problematic is determining enrollment 
statuf. for clock-hour students. These factors suggest that ED may want 
to issue clarifications concerning the determination c£ enrollment status 
for non-standard students. 

A subset of procedural or calculation errors appea's to be 
disproportionately distributed at a few institutions. For these 
institutions, sampling a relatively small group of recipients would 
uncover the occurrence of these systematic problems. Therefore, ED could 
investigate the characteristics of these institutions by drawing 
institution sariples. (Future evaluations of the Institutional Quality 
Control Pilot Project, where institutions draw a representative sanple of 
students, could be used for this investigation.) ED may also wish to 
design a sample in its audits of institutions that would indicate if 
institutions are having systematic problems with these items. 
Alternatively, ED could issue technical assistance or clarification 
concerning the problems with these items either through professional 
associations or through its own channels. 

Error in calculating EFC in the GSL program was primarily prevalent 
when institutions used the GSL Tables. Because recent changes made in 
the GSL program no longer allow institutions to use the GSL Tables, we 
suspect that a large portion of this error will be removed. 

Errors caused in the Campus-Based or GSL programs by institutions not 
estimating a Pell award for non-Pell recipients may be alleviated if ED 
stresses that the criteria for including P«ll, when determining need in 
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the Campus-Based or GSL programs, is strictly one of eligibility. 
Campus-Based or GSL applicants do not have to actually receive a Pell to 
use this v«lue when determining need. The reauthorization of the Higher 
Education Act did contain language stressing this fact. 

2.1 CORRECTIVE ACTIONS FOR SlGHIFICAiJT STUDENT ERRORS 

Student error can occur whenever any application value reported by 
the student diffei. from the best value obtained from the most reliable 
source during our data collection. This subsection presents the 
significant student errors across the three Title IV programs discussed 
in Findings and our proposed corrective actions for each error. In 
addition, we' will also discuss any relevant advantages or disadvantages 
that may apply to the corrective actions. 

There is a large am-unt of residual student error that is not 
amenable to the short-term or "quick-fix" corrective actions pro;?csed in 
the previous quality control studies. Much of this residual error cannot 
be corrected due to the complicated application process (e.g., multiple 
forms and instructions). Additionally, many of the residual student 
errors involve prospective information, such as expected income, 
household size, and number in college, that must be estimated at the time 
of application for the upcoming academic year. The prospective items are 
inherently error-prone, contributing to a large portion of the student 
error found in this and past quality control studies. Because 
prospective items cannot be verified, few corrective actions can be 
proposed that do not require changes in the current formulae and systems. 
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Corrective actions requiring these types of changes are discussed in 
later chapters. 



There are several application items, that proved to be significant 
sources of student error, where short-term, or Level I, type corrective 
actions are feasible: non-taxable income, home equity, number in 
college, household size, and dependent student's assets. Each of these 
significant student errors and our proposed corrective actions are 
discussed in the following sections. 

2*1.1 Incorrect Reporting of Other Non-Taxable Income 

The Application for Federal Student Aid (AFSA) requests information 
concerning sources of other base-year, non-taxable income and benefits 
(welfare, child support, worker's compensation, etc.). The applicant is 
to provide the total amount of income from such sources. For the 1985-86 
academic year, misreporting of other non-taxable income resulted in $75.1 
million in net error in the Pell program, and $23.7 million in net need 
error in the Campus-Based programs. Much of the error in misreporting 
other non-taxable income involved students (or their parents) who 
reported ze-^o other non-taxable income, when in fact they had such 
income. Table 2-1 presents the sources of other non-taxable income for 
the 21.6 percent of Title IV recipients erroneously reporting zero other 
non-taxable income. From this table we can see that untaxed portions of 
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TABLE 3-1 

SOURCES OF OTHER HOH-TAXABLE IHCOME FOR TITLE IV RECIPIENTS 
REPORTING ZERO OTHER SOH-TAXABLE INCOKE, 
1985-86 



Percent of those Reporting 
Source Zero with Positive Best Val ue 

Married Couple Deduction 50.5 

Untaxed Portions of Unemployment 17.4 

Interest and Dividend Exclusions 15.8 

Welfare 9.8 

Child Support 

Any Other Non-Taxable Income 2.6 
(e.g.. Black Lung Benefits, excess 
earned income credit, etc.) 

Veteran's Benefit 2.5 

Pension and Capital Gain 2.0 

Worker's Compensation 1-3 

Railroad Retirement Benefits O-^ 

Living Allowances 0-4 

Job Training Partnership Act 0-3 
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unemployment benefits and interest and dividend exclusions are the most 
frequently omitted source of other non-taxable income for applicants 
reporting zero other non-taxable income. 

ThG complexity of the application and instructions also contributes 
to this error. A student or parent completing this application could 
easily miss one or more of the income sources. Additionally, because 
these sources of income are non-taxable, they are not reported to the 
recipient on a W-2 or any other form. Therefore, there is a good deal of 
reliance upon the recipient's ability or willingness to recall these 
sources. 

Possible Corrective Actions 

Many possible sources jf non-taxable income are not reported to the 
recipient, and he or she may not have kept records of the income. For 
example, total child support may not be reported to an applicant and 
other records would be required to determine the full amount receive^u. 

One possible corrective action for reducing the error associated with 
reporting other non-taxable income is to cross-check the application with 
the Federal tax form for those items (i.e., a married couple deduction, 
etc.), that may be indicated on the tax form. 



Cross-checking other non-taxable income 
form would uncover discrepancies only in 



with items on the Federal 
the sources included on 



tax 
the 



Federal t«s retura. These items ar»{ a married couple deduction, 
interest and dividend exclusions, untaxed portions of pension and capital 
gains, and unemployment. However, starting with the 1987 Federal tax 
form, many of these non-taxable items will be deleted (e.g., married 
couple deduction) or they will be gradually phased out. Because of such 
changes in tax laws, there is a time limitation for this corrective 
action. 

2.1.2 Incorrect Reporting of Home Equity 

The AFSA requests information pertaining to the applicant's (and/or 
their parents-) net assets. Net assets include cash, savings, ana 
checking accounts, home equity, value of other real estate and 
investments. and value of businesses or farms. Of these various 
elements, home equity was most frequently misreported and resulted in a 
significant student error. For academic year 1985-86, misreporting of 
home equity resulted in $64.0 million in net error in the Pell program, 
and $22.9 million in net need error creating $16 million in awards in 
excess of need in the Campus-Based programs. Two types of error can 
occur with respect to reporting home equity, as follows: 

• The applicant reports zero home equity, when in fact there is 
home equity; or 

• The applicant reports home equity, but the reported amount is 
more or less than che actual home equity. 
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As isdic»ted earlier,- 15*4 percent of dependent students and 6.5 
percent of independent students who reported xero home equity actually 
had a best value for home equity. For those applicants who reported zero 
home equity, we checked to see if they claimed any home mortgage interest 
deduction on their IRS Schedule A. The results of this comparison are 
presented in Table 2-2. 



Possible Corrective Actions 



Underestimating the true value of the home might occur because the 
applicant does not use the correct reference to determine the true value 
(i.e., uses the tax value or insured value rather than current market 
value). Short of obtaining an assessment for each home, there is little 
that can be done to remove this problem. 

However, a corrective action that could be implemented to reduce the 
error associated wii-h homeowners reporting zero home equity is to 
cross-check the presence of home equity reported on the application with 
any amount of home mortgage interest claimed on Schedule A of the IRS tax 
form. As shown in Table 2-2, the majority of parent's of dependent 
students (59.9 percent ) and independent students ( 84 . 6 percent ) who 
reported zero home equity and claimed a home mortgage interest deduction 
on their Federal tax returns, actually were found to have home equity 
preSijnt using best values. This means that claiming mortgage interest is 
a good indicator that a home should be listed on the application. Not 
all persons having a home will claim a mortgage interest deduction, 
either because they do not file taxes, do not itemize deductions, or do 



TABLE 2->2 



DISCREPAMCIES IM REPORTING SELECTED ASSET ITEMS AND IRS VALUES 
rOR TITLE IV RECIPIENTS REPORTING ZERO^ 1985-86 



ITEMS FOR WHICH KtCiritWio 
REPORTED ZERO VALUES 


% WITH INTEREST 
ON TR^ FORMS 
HAVING BEST VALUES 


% WITH BEST VALUES 
CLAIMING INTEREST 
ON IRS FORMS 


Pdirenf s Home Equity 
(35% of ail recipients 
reported zero) 


59.9 


18.2 


Independent Student's 
Home Equity 
(87% of all recipients 
reported zero) 


84.6 


11.7 


Parent *s Savings 
(36% of all recipients 
reported zero) 


55.8 


25.6 


Dependent Student's 
Assets 

(71% of all recipients 
reported zero) 


46.8 


12.9 


Independent Student's 
Savings 

(50% of all recipients 
reported zero) 


51.2 


13.8 



For example: 59.9 percent of dependent recipients whose parents 
reported zero home equity and claimed mortgage interest on 
their Federal tax return had a best value for home 
equity. Hence, approximately 40 percent of dependent 
recipients whose parents reported zero home equity and 
claimed mortgage interest on their Federal tax return had 
a best value for home equity of $0. In addition^ 18.2 
percent of dependent recipients whose parents reported 
zero home equity and had a best value claimed mortgage 
interest on their Federal tax return. 




act pay interest on their home (i.e., it is fully paid off). However, 
reference to the tax form could still identify many cases (18.2 percent 
for dependent's parent's home equity and 11.7 percent for independent's 
home equity) where applicants report no home equity and actually do have 
home equity. While this only indicates the presence of a home, not the 
home equity, further questioning of the applicant would be required to 
determine the value. 

A drawback to this corrective action is that this cross-check can 
only be performed for tax-filers and persons who itemize deductions. 
Additionally, because the tax return provides information on the previous 
year and the applicant reports current home equity, discrepancies will 
naturally occur. For example, tha applicant may have sold his or her 
home prior to applying for financial aid. However, a comparison of the 
application and the tax return still can provide valuable information 
concerning those applicants who should be questioned further. 

2.1.3 Incorrect Beporting of Savings 

Applicants are required to provide information on the AFSA concerning 
the amount of money they or their parents have in a savings account (s) as 
of the date of application. For academic year 1985-86, misreporting of 
cash/checking/savings resulted in $1.5 million in net error in the Pell 
program, and -$2.8 million in net need error in the Campus-Based 
programs. Similiar to home equity, applicants may commit two types of 
errors when reporting this item: they report zero savings when they in 
fact have savings, or they report an amount different from what they 
actually have in their account(s). 



As stated earlier^ 2S.7 perctnt of independent applicants and 37,3 
percent of d«?endent applicants who reported sero savings actually had a 
best value for savings. We, therefore, compared zero reported savings 
with interest income on the Federal tax return in -an effort to identify 
those who erroneously report zero. The resvUts of this comparison are 
indicated in Tab. e 2-2. 

Possible Corrective Action 

Because the instructions appear to be quite clear as to what should 
be included in this item, additional instructions would not alleviate the 
problem of misreporting savings. Therefore, one possible corrective 
action that could be implemented to reduce error associated with persons 
reporting zero savings is to compare the amount reported for savings with 
the presence of income interest claimed on the Federal tax form. 

As shown in Table 2-2, the majority of parents of dependent students 
(55.8 percent) and independent students (51.2 percent) who reported zero 
savings and claimed earned interest income on their federal tax returns, 
actually were found to have savings present, using best values. As with 
home equity, this means that claiming earned interest is a good indicator 
that savings should be listed on the application. Not all persons with 
savings will have interest listed on their tax return. However, 
reference to the tax form could identify many cases (25.6 percent for 
dependent's parent's savings and 13.8 percent for independent's savings) 
where applicants report no savings and actually do have some amount of 
savings. This corrective action can only indicate that a savings account 
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may exist, not the amount of savings. Further questioning of the 
applicant would be necessary to determine the actual amount of savings. 

An additional disadvantage to this approach is that it proves only 
that applicants had savings in the year prior to application, not that 
they had savings at the time of application. However, if they reported 
zero savings and did claim earned income interest on their previous 
year's tax return, further documentation should be requested to verify 
the zero reported value. This approach does not provide a solution to 
misreporting by non-tax filers, 

2.1,4 Incorrect Reporting of Dependent Students* Assets 

Dependent applicants must provide information on the AFSA concerning 
their (and their spouse's) other assets besides the cash, savings, home 
value, already discussed. Some dependent students misreport assets by 
providing a value less than the true value of their assets, or by 
erroneously reporting zero assets. 

As indicated earlier, 31.3 percent of the applicants who reported 
zero dependent student's assets actually had best values. Again we 
attempted to analyze this problem by comparing zero reported dependent 
student assets with whether they claimed interest on their Federal tax 
returns. The result of the analysis is presented in Table 2-2, 
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Because of the similarity of this error with the previously discussed 
errors associated with assets (i.e., home value and savings) we suggest 
the same corrective action be applied to misreporting of dependent 
student's assets. Institutions should cross-check the value reported by 
the applicant with any interest claimed on the Federal tax form for 
selected students. As shown in Table 2-2, interest claimed is a good 
indicator of dependent's net assets since nearly half of the applicants 
(46.8 percent) who reported zero for dependent student's assets and had 
earned interest income on their tax forms, actually were found to have an 
amot-r- present using best values for this data item. Because many 
dependent students do not file tax returns, not all students with net 
assets will be identifiable in this manner. However, by referencing the 
tax returns, institutions could identify several cases (12.9 percent) 
where applicants have reported zero dependent student's assets and 
actually have assets. Again, this comparison would only indicate that 
some assets should have been reported on the application, not the value 
of the assets. Further questioning of the applicant would be required to 
determine the value. 

2.1.5 Incorrect ReTfCT*:^uq of Household Size 

At the time of application, students are required to project their 
family size for the upcoming academic year. Misreporting family size is 
a significant student error. As with expected income, estimating the 
far.ily size is inherently error prone due to the flexibility of family 
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plans and circumstances. This student-reported item was a significant 
erxror in the Title IV programs, resulting in $29.9 million in net error 
in the Pell program, and $58.7 million in net need error in the 
Campus-Based programs. In Pell, for example, incorrect household size 
was the sixth largest source of student error. 

For purposes of this study, verifying household size involved asking 
the student and/or parent (during a personal interview) what number they 
had reported on the application for expected household size, what the 
actual number was, and the reason(s) for any changes. We also abstracted 
household size information from documentation contained in the student's 
institutional fi)8. 

In order to analyze errors in household size, we compared the number 
on the application the student stated they expected to be in the 
household during the school year to the actual number in the household 
during the year. Insight into possible corrective actions is provided by 
comparing these values to the student's statement concerning an 
unanticipated change in household size. 

Table 2-3 presents the percentage of recipients who misreported 
family household size by whether there was or was not an unanticipated 
change in family size. As shown in Table 2-3, 90.2 percent of the 
recipients had no unanticipated change, i.e., there was no change between 
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TABLE 2-3 

CHAKGES IN REPORTED BOUSEBOLD SIZE 
(MOT IHCLODIMG UPDATES), 
ALL TITLE IV PROGRAKS, 1985-86 



REPORTED LESS 
THAN BEST VALUE 
(PZRCENT) 



REPORTED GREATER 
THAN BEST VALUE 
(PERCENT) 



No Unanticipated Change 
(90.2% of Recipients) 



70,4 



60.4 



Unanticipated Change 
(9.8% of Recipients) 



TOTAL 



29.6 



100.0 



39.6 
100.0 



Independents 



No Unanticipated Change 
(92.6% of Independents) 



66.1 



59.9 



Unanticipated Change 
(7.5% of Independents) 



TOTAL 



33.9 



100.0 



40.1 



100.0 



Dependents 



No Unanticipated Change 
(88.6% of Dependents) 



73.4 



60.5 



Unanv ^'^ated Change 
(11.4% or dependents) 



TOTAL 



26.6 



100.0 



39.5 
100.0 



For example: 



70 4 percent of recipients whose reportea household size 
was less than their best household size did not have an 
unanticipated change that could have explained the 
dif fer-snce. 
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the household size the student anticipated for the academic year, and 
what actually occurred. An unanticipated change occurs when, for 
example, a family member moves in or out of the household, a divorce, a 
separation, or a death occurs. Dependent students have a slightly larg^-r 
percentage (11.4 percent) of recipients who did have an unanticipated 
change in household size as compared to independents (7.5 percent). 

Ot the recipirits who reported a household sir, value less than the 
best value, 70 percent of the recipients did not have unanticipated 
change in their household size. This type of error, because it works 
against the student, most probably is the result of careless completion 
of the application or not understanding the question (e.g., not including 
the applicant, another student, grandparents, etc., who are living with 
the family) . 

Of the recipients who overestimated their household size (i.e., 
reported values were greater than best values), almost 40 percent 
experienced unanticipated changes in their household size between the 
time of application and verification. The remaining 60 percent who 
reported a household size greater than best value had no unanticipated 
change in their household. Thus it ii not true that household size error 
is caused by circumstances beyond the control of the recipient. This is 
true for recipients who report a household size less than best as well as 
for recipients who report a household size greater than best. 

Although the percentage of independent and dependent students who had 
unanticipated changes in household size was quite small, each made a 
large contribution to over-reporting error. The 7.5 percent of 



independent students with unanticipated changes contributed 40.1 percent 
of the error attributable to repotting a household size larger than the 
best value. Similarly, the 11.4 percent of dependent students who had 
unannicipated changes in household size contributed 39.5 percent of the 
error for reporting a value greater than the best value. 

For independent recipients who had an unanticipated change in their 
household size, the most commonly cited reason was having been married 
after applying for financial aid. For dependent recipients with a change 
in household size, a household member moving out of the household was the 
most frequent change. Independent and dependent recipients' responses 
for the primary reason for an unanticipated change in household size are 
presented in Table 2-4. 

Possible Corrective Actions 

There is some difficulty in projecting household size into the 
upcoming academic year because the application is completed prior to the 
start of the academic year and updated at the time of enrollment. 
Additionally, family circumstances and plans can change rapidly, 
affecting this prospective data item as is indicated by the large 
proportion of household size misreporting attributable to unanticipated 
-hanges. There are three proposed corrective actions that may reduce the 
error associated with reporting on this data item. They are the 
following: 

• Change the definition of household size to equal number of 
exemptions . 
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TABLE 2-4 



PRIMARY REASON CITED BY RECIPIENTS 
FOR UNANTICIPATED CHANGES IN HOUSEHOLD SIZE 
IN ALL TITLE IV PROGRAMS, 1985-86 



Reason For Change In 
Household Size 

New Addition to Family 
Got Married 
Someone Moved Out 
Someone Moved In 
Support Plans Changed* 
Se-^arated/Divorced 
Death in Family 
No Answer 



Recipients With Unanticipated Changes 
I ndependent Dependent 
(Percent) (Percent) 



15.5 


10. 


0 


28.5 


23. 


0 


17.5 


36. 


4 


16.8 


7 


0 


0 


4 


.6 


18.3 


11 


.9 


1.9 


5 


.2 


1.5 


1 


.8 



♦Change in plans to provide or not provide at least one-half suppo 
to others. 
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• Require that institutions obtain a completed Verification 
Worktheet from each applicant, which requests a listing of 
household members by name, age, relationship, and the name of 
the college they attend, if applicable. 

• Use the number of exemptions claimed on the Federal tax 
return as a verification edit when exemptions are smaller 
than household size. 

Change the Defiuition of Household Size to Equal the Number o f Exemptions 

Given the inherent difficulty in projecting household size, a 
corrective action would be to use base-year data (i.e., the number in the 
household at the time of application) to determine household size. Using 
the number of exemptions reported on the Federal tax return as the number 
in household results in values similar to the number in the household at 
the time of application. (We did not collect data on the actual number 
in house old at the time of application because this value was not used 
in determining awards under the current program specification. Rather, 
we v:oilected and used the number at the time of application, that the 
student expected to be in the household during the school year.) We 
performed a simulation comparing best values with reported household size 
and number of exemptions. The results are presented in Table 2-5. 

The reported values were equal to best values for 77 percent of 
applicants, while the number of exemptions was equal to best values for 
68 percent of applicants. The main problem with using exemptions is that 
a larger percent of students (19 percent) would have the amount of aid 
awarded based on a household size smaller than it would have been. When 
using reported values, only 8 percent of students would have a value less 
than best values. 
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TABLE 2-5 



ACCOBACY OF REPORTED BOUSEBOLD SIZE AHD TAX FORM EXEMPTIONS 
AS COMPARED TO BEST VALUES 
ALL TITLE IV PROGRAMS, 1985-86 

REPORTED LESS REPORTED EQUAL REPORTED GREATER 

THAN BEST VALUE TO BEST VALUE THAN BEST VALUE 

(Percent) (Percent) (Percect) 



Reported 8 77 15 

Household Size 

IRS Exemptions 19 68 



FRir 
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Require That Institutions Obtain a Complete d Verification Worksheet From 
Each Applicant 

This corrective action is directed at those students who had 
anticipeted a change in household size but still misreported household 
size. Updating, by asking if anything has changed, does not solve the 
initial misreporting, A more comprehensive and detailed listing of 
household size might help reduce error for these persons. The 
Verification Worksheet contains a section where applicants are to provide 
information concerning the people in their household including their 
name, age, relationship and the college they attend if applicable. An 
advantage to this approach is that the Verification Worksheet is an 
existing document and would not require any revisions, A disadvantage is 
that currently, only those students flagged for verification complete 
thete forms if reque-.ted by their institution. Currently, the 
institutions have the choice of using the Verification Worksheet or a 
worksheet they have developed instead. This approach would require that 
all institutions use the Verification Worksheet, removing all discretion 
from the institution. The burden associated with this corrective action 
would be extremely high and might not be allowed under reauthorization. 

Use the Exemptions or the Federal Tax Return as a Verification Edit 

The Federal tax return could be used as an edit cor verifying 
household size when the number of exemptions on the tax return is smaller 
than the household size reported on the student's application. Selecting 
applications to verify according to this criteria would identify those 



applications where it was most likely a serious error in household size 
occurred, and where a potentially large overpayment could result. 

2.1.6 Incorrect Reporting of Humber in College 

Title IV applicants are required to report the number of household 
members who will be enrolled in postsecondary education. Misreporting of 
number in college resulted in $18.4 million in net error for the Pell 
program and $26.5 million in net need error for the Campus-Based 
programs. Misreporting resulted more often in overstatements of need in 
the Campus-Based programs and more overawards for the Pell program. 

Table 2-6 presents the percentage of recipients who misreported 
uumber in college by whether or not there was an unanticipated change in 
the number of household members attending college. As shown in 
Table 2-6, 94.4 percent of the applicants had no unanticipated change 
between the number in college the student anticipated for the ac^'iemic 
year, and what actually occurred. Of the recipients who reported a 
number in college lower than the best value, 22.6 percent experienced 
unanticipated changes. Of the recipients who overestimated the number in 
college (i.e., the reported value was greater than the best value), 41.0 
percent had unanticipated changes that affected tlie number in their 
household in college. Dependent students have a slightly larger percent 
(6.9 percent) of recipients who had an unanticipated change in the number 
in college as compared to independents (3.6 percent). 
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TABLE 2-6 



CHANGES IN REPORTED NUMBER IN COlOliEGE, 
ALL TITLE IV PROGRAMS. 1985-86 

\ 

T 

REPORTED LESS REPORTED GREATER 

THAN BEST VALUE THAN BEST VALUE 
(PERCENT) (PERCENT) 



No Unanticipated Change 

(94.4% of Recipient) 77.4 59.0 

Unanticipated Change 

(5.6% of Recipients) 22.6 41.0 

TOTAL 100.0 100.0 

Independents 

No Unanticipated Change 

(96.5% of Independents) 47.8 59.6 

Unanticipated Change 

(3.6^ of independents) 52.2 40.4 

TOTAL 100.0 lOO.O 

Dependents 

No Unanticipated Change 

(93.1% of Dependents) 89.8 58.9 

Unanticipated Change 

(6.9% of Dependents) 10. 2 41.1 

TOTAL 100.0 100.0 



For example: 77.4 percent of recipients whose reported number in 
college was less than their best number in college did not 
have an unanticipated change that could have explained the 
difference . 



Possible Corrective Actions 

As indicated in Table 2-6, 22.6 percent of recipients reporting less 
than best val"^ and 41.0 percent reporting greater than best value had an 
unanticipated change in the number of household members in college. Both 
are significant percentages of error. Because unanticipated changes 
cannot be controlled, the only corrective action that might reduce these 
types of errors would be to use the base-year values (i.e., the value at 
the time of application) for number in college. A simulated comparison 
between base-year and best values for the current year should be made to 
determine the accuracy of base-year data. We do not have data for 
base-year number in college since it is not part of the current award 
determination and, therefore. cannot make this comparison. As with 
household size, we propose corrective actions aimed at reducing error 
associ?ted with studenti-. hewing no unanticipated changes. 

The question and instructions on the application, about number in 
college, are uncomplicated. This results in few questions about how to 
include dependent children who are enrolled at least half time. 

Suggested corrective actions to address the error associated with 
reporting the number in college are the following: 



• Require that institutions obtain a completed Verification 
Worksheet from each applicant which requests a listing of 
household members by name, age, relationship, and the nanie of 
the college they attend, if applicable. 

• Verify all applicants who report more than one (including the 
applicant) in pos tsecondary education. 
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- mv^-^ i 4...*. irt-*- OK**;,^ * pomniAhAd VAfification Worksheet From 

Each Applicant 

This is the same corrective action discussed in the previous 
section. 

Verify All Applicants Who Report More Than One in Pos t secondary 
Education 

Another approach to reducing the number in college error is verifying 
all applicants who report more than one family member in college. 
However, this would have required that for the 1985-86 academic year, 
1,344,852 (or 30 percent) of the applicants be verified for number in 
college. ED will have to consider whether they want to place this burden 
on the institutions and the applicants. 

2.2 CORRECTIVE ACTIONS FOR SIGNIFICANT INSTITUTIONAL ERRORS 

The analyses presented in Findings indicated that institutional 
errors were significant in the Pell, Campus-Based, and GSL programs. In 
many cases, while the frequencies of institutional errors were low, the 
effects of these errors were large. Those institutional errors that were 
significant in Findings , and that we discuss in this section, are the 
following: 

• Enrollment status error in the Pell Grant program, which 
occurred for 18.2 percent of Pell recipients totalling $100.9 
million in underawards and $110.5 million in overawards; 

• A composite index of Pell calculation error, Campus-Based 
initial overawards and disbursement error, and GSL initial 
overawards and errors in factoring other aid; 



♦ EFC error in the GSL program (6*2 percent of GSL 
certifications and $260-0 million in certification error); and 

• Institutional Campus-Based and GSL errors caused by errors in 
handling Pell awards (22.7 percent of Campus-Based recipients 
for $174,5 million in absolute need error and 3,8 percent of 
GSL certifications for $84,6 million in certification error). 

There are several basic types of corrective actions that can be 
considered for these types of errors. These corrective actions include 
clarifications issued by ED for procedures institutions should follow, 
providing technical assistance tools to help schools conduct procedures 
more accurately, and modifications to the procedures institutions follow 
in order to make them simpler and less error-prone- The corrective 
actions for the institutional errors identified above include all three 
of these. The errors mentioned above, and corresponding corrective 
actions are discussed in the following sections. 

2. 2*1 Pell Enrollment Status Error 

Tables 2-7 through 2-9 present a breakdown of Pell enrollment status 
error by three factors that would likely affect an institution's ability 
to calculate their students' enrollment status correctly. The factors 
are the credit system used by the institution, summer session attendance, 
and changes in enrollment status. The results generated when analyzing 
enrollment status error by these three factors suggest that higher rates 
of enrollment status error are prevalent in situations where calculating 
the proper enrollment status is more complicated than usual* 
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Table 2-7 shows that students at clock hour schools ars more likely 
to have an enrollment status error than students at credit hour 
institutions. In particular, this problem is noticeable in underawards 
where the error rate for students at clock hour schools is nearly four 
times the rate of enrollment status underawards for students at credit 
hour institutions. This result is not surprising given that the vast 
majority of recipients attend a credit hour school and, consequently, the 
regulations are directed at credit hour schools. Attempting to fit clock 
hour schools into the framework developed for credit hour schools results 
in confusing and hard to follow regulations. 

Table 2-8 shows that students enrolled in a program or institution 
that required them to attend a summer session had an enrollment status 
error 61.7 pi.-rcent of the time. Students who attended a summer session, 
but were not required to, had a 28.5 percent error rate, while students 
who were not required to and did not attend summer sessions had a 13.6 
percent error rate. The high error rate for students required to attend 
summer sessions results from a high rate of overawards. This indicates 
that institutions are not reducing Pell awards in situations where 
students do not attend required summer sessions or where their attendance 
is at a reduced level compared to the regular-term sessions. The high 
rate of underawards for students attending optional summer sessions means 
that institutions may not be allowing attendance at summer sessions to 
increase awards in cases where the student had not yet received their 
scheduled full-time award. High error rates associated with summer 
sessions may also occur because summer ^cions are usually structured 
very differently than regular term sessions, so that calculating 
enrollment status properly can prove difficult. 
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TABLE 2-7 

KMROLLMENT STATUS ERROR BY CREDIT SYSTEM, 
1985-86 



NO ERROR 

(Within $50) UNDERAHARD 

(Percent) (Percent) 



O^nSRAWARD 

(Percent) 



Credit Hours 

(88. 7\ of Recipients) 

Clock Hours 

^11. 3% of Recipients) 



83.6 



66.8 



6.4 



23.4 



10.0 



9.8 



TABLE 2-8 

ENROLLMENT STATUS ERROR BY SU>»4ER SESSION ATTENDANCE, 

1985-86 



NO ERROR 

(Within $50) UNDERAWARD OVERAWARD 

(Percent) (Percent) (Percent) 



Summer Session Required 
(3.5S of Recipients) 

Summer Session Optional, 

Student Attended 

(7 . 2% of Recipients) 

Summer Session Optional, 
Student Did Not Attend 
(89. 3% of Recipients) 



38.3 



71.5 



86.4 



9.1 



23.3 



5.0 



52.6 



5.2 



8.6 



Finally. Table 2-9 shows that students whose enrollment status 
changes, and part-time students, have significantly higher rates of 
enrollment status error than full-time students whose enrollment status 
does not change. In particular, the high rate of overawards for students 
whose enrollment status changes indicates that students decrease 
enrollment status more often than increase. Enrollment status error is 
significantly higher for part-time students and students who, for one 
reason or another, have changes in their enrollment status. For these 
students, enrollment status is more complicated to calculate. 



Possible Corrective Actions 



The analyses of Pell enrollment status error suggest that, in 
general, institutions determined enrollment status relatively well. Only 
when institutions were determining enrollment status for non-standard 
students (i.e., students at clock hour schools, students who attended a 
summer session, students who did not attend a summer session but were 
required to, and part-time students or students whose enrollment status 
changed) did error rates significantly increase. This pattern would 
suggest that ED might need to take several possible corrective actions 
including the following: 



Clarify the procedures for determining enrollment status at 
clock hour institutions or simplify the process for doing so 

• Instruct schools on the types of situations or students that 
are likely to have an enrollment status that is difficult to 
determine, and 
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TABLE 2-9 



ENROLLMENT STATl?S ERROR BY CHANGES IN ENROLLMENT STATUS* 

1985-86 



NO ERROR 

(Within S50) UNDERAWARD OVERAWARD 
(PERCENT) (PERCENT) (PERCENT) 

Full-Time Students With No 94.0 4.6 1.4 

Changes in Enrol Irr.ent Status 
(67. 1*^ of Recipients) 



Part-Time Students With No 61.0 20.3 18.7 

Changes in Enrollment Status 
{4.0*^ of Recipients) 



Full-Time and Part-Time 52.6 28.5 

Students With Changes in 
Enrollment Status 
(28.9% of Recipients) 
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• Clarify the allowances for summer session attendance in the 
cases where summer session is not required and also where 
schools are not combining properly the required summer 
session enrollment status with the regular term enrollment 
s tatus. 

The patterns of error for enrollment status are not new. 
Historically, determining enrollment status at clock hour institutions or 
for students with status changes has been difficult. These historical 
patterns may suggest that any corrective actions short of restructuring 
the method of determining enrollment status might be of limited impact. 
For example, given the extensive problems in determining enrollment 
status in summer sessions, it might be better to calculate enrollment 
status based just on regular-term sessions by not allowing for 
adjustments based on samr»er session. 

2.2.2 Composite Analysis of Procedural Errors 

We have combined five institutional errors (Pell calculation errors, 
Campus-Based initial overawards. Campus -^Based disbursement error, GSL 
initial overawards, and errors in factoring other aid in the GSL program) 
in one composite analysis. These five errors are similar in that they 
all relate to problems in disbursing aid or coordinating all sources of 
aid. While the errors occur relatively infrequently, when they do occur 
they tend to have relatively large payment consequences and so are worth 
cons i^.^e ring. 

The relatively low occurrence of these errors posed difficulties for 
our analysis. The fact that the error rates were so low meant that 
breaking down the error rates by meaningful factors was nearly 



I 



z 

f, 

impossible. Because of this, bivariate and multivariate analyses were { 
not feasible. 

I 

In orier to analyze these errors as a group, we developed an index i 

i 

that enabled us to examine the extent to which the errors clustered at 
institutions to see if a few institutions might be causing a 
disproportionately large amount of the errors. Our initial analyses 

using this composite index suggested that while many schools had no ^ | 

i 

errors of this type, a few schools had systematic problems in this area. I 

These few schools were associated with an unusually large proportion of 

the errors. Exhibit 2-1 summarizes the results of our analysis. This 

exhibit shows that 42.4 percent of the schools had no sampled students 

with any of these errors. At the other extreme, 5.4 percent of the 

schools had 50 percent to 100 percent of the sampled students with at 

least one of the five errors. These results, while suggesting an 

underlying relationship, do not confirm the existence of a statistical 

relationship. We cannot infer a statistical relationship because we drew 

a student sample, not an institution sample. Sample sizes at individual 

institutions were simply too small to estimate institution-specific exror 

rates . 



Possible Corrective Actions 



This group of composite errors is a significant component of 
institutional error in the Pell, Campus -Based , and GSL programs. In 
spite of this, because of the low incidence of these errors and because 
we did not draw institution samples, developing corrective actions for 
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42.4% 
nMl25 




P«rc«nt of 
Intlitutiont 
(n = 295)* 



; School had no respond$nt$ (9ach $mnphd studmt <Md not respond) mid 2 schools woro combinod 

becauss there was only 1 lO fa botfi schools. 

EXHIBnr 2-1. DISTRIBUTION OF ERROR RATES 
FOR COMPOSITE INDEX OF PROCEDURAL ERRORS 

70 



these errors is extremely difficult. However, because our analyses 
suggested that these errors may be disproportionately distributed across 
institutions, and because our analyses in Chapter 5 suggest some strong 
relationships with institutional quality control procedures, the 
Institutional Quality Control Pilot Project, where schools draw 
institutional samples, could be used in future Pilot years to further 
examine this composite group of errors. Given the systematic nature of 
the errors at the schools where they do occur, a relatively small sample 
could identify the existence of these types of disbursement and 
coordination problems. A small sample designed to uncover these errors 
could be incorporated in ED's audits of institutions. Alternatively, ED 
could develop technical assistance materials on how schools could 
identify these errors when they occur systematically. These technical 
assistance materials could be based, in part, on the sampling procedures 
contained in the Pilot project. The technical assistance materials could 
be distributed through professional associations (which could also focus 
on training institutions in detecting these errors) or through ED. 



2.2.3 GSL EFC Error 



Error in determining the correct EFC was significant in the GSL 
program. This error occurred primarily in cases where schools used the 
GSL Tables to determine the EFC. Table 2-10 shows that students whose 
EFC was determined using the GSL Tables had a GSL EFC error 26 percent of 
the time compared to 8 . 5 percent for Campus-Based need analysis and 0.3 
percent when the AGI was less than $30,000. Error in the GSL Tables 
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TJIBLE 2-10 



INSTIIUTIOKAL GSL EFC ERROR 
BY WFE OF HEED ANALYSIS 
SYSTEM, 1985-86 



MEED SYSTEM 



Percent 



EFC ERROR 



Total 

Mean (S) (S Millions) 



Campus-Based Need 
Analysis 

(7.3% of Recipients ) 
GSL Tables 

(20.8% of Recipients) 

AGI Under $30,000 - No Need 
Analysis 

(72. OS of Recipients) 



8.5 



26.0 



0.3 



955 



1, 199 



1,701 



20.6 



226.7 



12.7 



accounted for $226*7 million of the $260 million in total GSL EFC error. 
Recent changes made in reauthorization of the Higher Education Act will 
require all GSL applicants to go through full need analysis and do not 
allow schools to use the GSL tables • Because of this change, we 
anticipate that the majority of the GSL EFC error will be removed. 

There are several possible reasons why the GSL Tables could be 
associated with such a significant portion of the GSL EFC error. These 
possibilities include the following: 

• Institutions misread or misused (e.g., net prorating 
properly) the GSL Tables, thus determining an incorrect EFC; 

• Institutions adjusted the value they read from the GSt 
Tables, causing an incorrect EFC; or 

• An institution stated it used the GSL Tables to determine EFC 
when, in fact, the institution used another method. 

Any of these reasons would cause an error, and all three likely 
contributed to the large amount of GSL EFC error. 

2.2.4 Campus-Based and GSL Institutional Errors Caused by Errors in 
Handling Pell Awards 

Federal student financial assistance is designed so that students' 
first level of support comes from the Pell program. Other programs 
should only be used if there is need remaining after the Pell Grant (to 
which the student is entitled) has been subtr-r^cted from cost. 
Consequently, the incorrect handling and distribution of Pell Grant 
awards can cause further errors in both the Campus-Based and GSL programs. 
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In Findings , errors in factoring Pell awards were identified as 
having a large marginal impact on institutional error in both the 
Campus-Based and GSL programs. The error was defined as the difference 
between the Pell award used by the institution in determining 
Campus-Based need or the GSL certification amount, and the award that 
should have been used based on reported SAI, best cost of attendance, and 
best enrollment status. In the Campus-Based programs, error in factoring 
Pell awards was the largest institutional marginal error in terms of both 
percentage of cases in error (22.7 percent) and absolute need ($174.5 
million). However, its impact on awards in excess of need was far lower, 
only $2.9 million. Error in factoring Pell awards was the fourth highest 
GSL institutional marginal dollar error, occurring in 3.8 percent of 
cases for a total of $84.6 million. 

Errors in factoring or handling Pell awards can be caused by three 
factors: 

• An institutional error committed when disbursing the Pell 
award 

« A difference between the Pell disbursement amount and the 
award used to calculate need or certification, or 

© Failure to subtract from cost the Pell a student was entitled 
to but did not receive in determining need for the 
Campus-Based and GSL programs. 

Institutional errors made during Pell awards which affect the other 
programs are merely a function of the interrelationship between the 
programs. Thus, these errors are not actually a Campus-Based or GSL 
problem. Differences between the Pell award actually disbursed and the 
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award used in calculating need is probably caused by changes in Pell 
occurring during the award year. Since need is usually determined at the 
beginning of the year, any change in trhe expected Pell award can lead to 
differences between the Pell disbursed and the Pell used. 

This error becomes a problem only if the difference caused need to 
fall below award or ce fication amount. Changes occurring after the 
beginning of the progrant year that affect only the amount of excess need 
can be ignored since it is unlikely that schools would repackage awards 
based on the new need during the year. The fact that there was a small 
amount of awards in excess of need associated with errors in factoring 
Pell awards, even though there was a high amount of need error, may 
indicate tha> in the Campus-Based programs institutions are accounting 
for Pell changes that cause need to fall below award but are ignoring 
those that affect only unmet need. 

The errors discussed above relate primarily to Campus-Based and/or 
GSL recipients that also receive a Pell award. Because Pell is always 
supposed to be awarded before other Title IV aid, institutional errortr in 
factoring Pell awards can also occur for Pell non-recipients. Table 2-11 
demonstrates that in the Campus-Based programs, 9.1 percent of Pell 
non-recipients were in fact eligible to receive Pell awards, bdsed on 
reported data. Of these 9.1 percent, institutions comxritced an error 
69.2 percent of the time by failing to include the Pell award to which 
the student was entitled in the calculation of need. This failure caused 
an estimated S31.7 million in need overstatements which account for 18.2 
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ERfiOSS IM FACTORIHG PELL AWARDS 
IN THE CAMPUS-BASED AMD GSL PROGRAMS 
FOR HOH-PELL RECIPIEKTS, 1985-86 



PERCEirr 

PELL ELIGIBLE 



ERROR AMOWG PELL ELIGIBLES 



Percent 



Total 

Mean ($) « Millions) 



Campus -Based* 9.1 
GSL 8.0 



69.2 
14.6 



1,187 
751 



31.7 
19.9** 



* Error results in overstatements of need only. 

♦* Due to a revision in the estimate of total GSL loan volume, these 
should be reduced by approximately 10 percent, 
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percent of the absolute need error and 44.7 percent of the need 
overstatements associated with errors in factoring Pell awards. However, 
the awards in exc«ss of need was far smaller. The comparable figures for 
GSL are that 8 percent of Pell non-recipients were eligible to receive 
Pell awards with institutions failing to consider their Pell eligibility 
in calculating the GSL certification 14.6 percent of the time. This 
resulted in $19.9 million in GSL institutional certification error, 23.5 
percent of the error associated with incorrectly factoring Pell awards. 

Possible Corrective Actions 

The figures discussed above indicate that institutions, particularly 
in the Campus-Based programs, are not properly esse<;sing Pell eligibility 
before calculating need. Most likely the institutions are relying on the 
Pell award the student is actually expected to receive to determine their 
Pell eligibility for inclusion in the other Title IV programs. It must 
be impressed upon institutions, therefore, that the criteria for 
including Pell in the determination of need for the Campus-Based and GSL 
programs is eligibility and not the actual receipt of an award. 
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3.0 

AHALYSIS or PROSPECTIVE INCOME DATA 

Recent changes made by Congress in the Title IV programs do not allow 
for prospective income data to be used in determining either awards in 
the Pell Grant program or need in the Campus-Based programs except for 
dislocated workers. Changing the treatment of prospective income items 
is a Level II corrective action. This corrective action will affect 
larger aspects of the delivery system as well as a larger portion of 
residual error than the corrective actions for the data items presented 
in Chapter 2. This chapter presents our analyses of the likely effects 
of the change from prospective to base year income in the Pell and 
Campus-Based programs. Our findings indicate the following: 



• Campus-Based need using base year income data will be 
substantially different than Campus-Based need using 
prospective income data. 

• Both the apparent and actual changes in need resulting from 
the move to base year income data cause a downward shift in 
need in the Campus-Based programs. 

• Independent students* need error rates in the Campus-Based 
programs attributable to income using base year data are less 
than half of the error rates using prospective ^dta. 

• Fewer than 20 percent of Pell recipients are affected by the 
change to base year income data (not including Special 
Condition filers). 

• The current method of selecting which Pell recipients should 
have their awards based on prospective income is in error 
approximately 29 percent of the time (15.7 percent of Pell 
recipients use prospective income when they should not and 
13.1 percent do not use w^en they should). 

• The change affects Pell recipients with low awards the most. 

• Dependent student income error in the Pell program using base 
year data declines approximately 25 percent (from 16 to 12 
percent) . 
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change is likely to be judged favorably in spite of the other 

distributional effects. The above findings are presented in detail in 



the following sections. 
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Prior QC studies have indicated that prospective data in general, and 
specifically prospective income items, are highly error-prone and result 
in considerable payi^ent or need error. Current Federal and other 
applications require applicants and fan.ilies to estimate data for the 
year in which they will receive student aid. Studies have shown that 
these estimates are, at best, subject to changes in circumstance over 
time (as suggested in the preceding chapter) and, at ..orst, to conscious 
manipulation or misestimation. 

Prospective items used in the current formula include household size 
and number in college for all applicants, income for certain dependent 
students applying for Pell Grants, and taxable and non-taxable income for 
all independent students applying for Campus-Based assistance. 
Prospective income has been one of the largest sources of error in the 
Campus-Based programs despite the fact that fewer than half of all 
Campus-Based recipients are independent. 

With the recent reauthorization of the Higher Education Act, Congress 
altered the formula for determining Campus-Based need for independent 



applicants and for determining Pell Grant eligibility for certain 
dependent students The Title IV Quality Control Project provides a 
unique data base with which to con-uct analyses of the effects of 
changing from prospective to base year income data because the data base 
contain both reported and best values. Other analyses have been 
conduc-.ed using reported prospective and base year income. These 
reported data are subject to error and thus analyses using these dat.' can 
only estimate the apparent effects of the change. The Title IV QC data 
base can estimate both the apparent and true effects. True effects can 
be estimated by using best data in the analysis, thus removing from the 
analysis distortions caused by reporting error. 

Clearly, the changes mandated by reauthorization will affect the 
Uniform Methodology (UM) to a greater extent than the Pell Grant Family 
Contribution Schedule (FCS). The changes will affect the UM need in the 
Campus -Based programs for independent students, which represent 

slightly less than one-half million recipients. The changes will also 
affect nearly 20 percent of the 2.8 million Pell recipients. These 
changes will have potential impact on a large number of students. Thus 
the analysis presented here addresses three questions that are 
particularly relevant to policy makers: 

• Does the base year income data achieve intended effects 
better than prospective year data? 

• How are current recipients affected by these changes? 
« How will the error rate be affected by these changes? 



we approach this analysis with the assumption that Congress altered 
the UM for independent students and the FCS for dependent students in 
response to the pervasive problem of high error associated with 
prospective income. There is no indication that Congress acted with 
other motives such as a desire to change the f undair.ental income measure 
on which awards are based. This motivation suggests a testable research 
question, i.e. will base year data result in a distribution of UM need or 
Pell awards that is closer to the intended distribution (as produced by 
best prospective data) than reported prospective data. This question can 
be addressed in the Campus-Based programs by ranking the alternatives 
(e.g.. base year reported or base year best) on the basis of their 
deviation from UM need using the best prospective model. 

Unlike the UM, the Pell program uses a "trigger" to select the cases 
for which prospective data will be used. The trigger selects students 
determined by their best prospective year income data. Because of this 
aspect of the Pell program, a different analysis is appropriate. Rather 
than ranking, we cai address the question of whether the prospective 
income "trigger" is accurately selecting dependent students for whom 
prospective income should be used. 

In addition, the effects of the changes in the Pell and Campus-Based 
programs can be described by analyzing the shifts in need, by level of 
need or award. This analysis wii; identify the subpopulations that are 
affected most. Lastly, we examine the likely impact on error of the 
changes in determining UM need or Pell award. 
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3.2 KrrECTS OH CAMPUS-BASED RECIPIBMTS 

The effects of changing to base year income in the UM can be 
determined most meaningfully by using Campus-Based need to measure the 
change. We estimated need changes by holding all other components of 
need (e.g., cost, other aid, etc.) constant at reported values and 
replacing the original UM EFC with an EFC recalculated using base year 
data for independent students. Thus, the changes analysed in this 
section are the result of EFC changes. The following suDsections deal 
with each of the three policy questions listed above. 

3.2.1 Ranking the Models 

Using an index of deviation from need determined with best 
prospective data, we can rank each of the models (e.g. reported 
prospective, best base year, and reported base year) to determine which, 
alternative results in the smallest differences. This ranking will 
determine whether replacing prospective income with base year moved the 
UM closer to the distribution of need intended under the current 
formula. Table 3-1 below presents the results of this ranking. 

TABLE 3-1 

RANKING THE MODELS BASED OM DISTANCE 
FROM BEST PROSPECTIVE NEED, 1985-86 

MODEL DISTANCeI 
Reported Prospective ^''"'^ 
Best Base Year '^'^^^ 
Reported Base Year ^258 

1 Appendix A describes, in detail, arithmetically how these distances 
were computed. 



The above differences can be interpreted as weighted mean absolute 
dollar differences in need for independent students. These differences 
indicate that the difference between the distribution of need obtained 
with best prospective and reported prospective data is smaller than for 
either best or reported base year income. This suggests that while 
substantial differences exist between all the models and the best 
prospective model, the model using reported prospective data is slightly 
superior to the other models in achieving the distribution of need 
intended under the current UM. 

3.2.2 Effects on Current Recipients 

The UM uses prospective income to determine need for independent 
students because of potentially large year to year differences in 
income. Thus, the effects will be greatest for those independent 
students whose income changes substantially between the base year and the 
prospective year. Analysis of the likely effects resulting from the 
shift from prospective to base year income for independent students must 
distinguish between the apparent and the truT effects. Apparent effects 
result from a comparison of base and prospective models using reported 
data. These are not true effects since these data contain reporting 
error that distorts the distribution of need in the population. Thus, 
true effects can only be measured when best data are used. 



Exhibit 3-1 displays the apparent effects using base' year reported 
data to datermine need for independent students. (Table B-1 in Appendix 
B provides additional information.) This figure presents the percentage 
of recipients according to the need they had under the two models. 
Exhibit 3-1 indicates that the majority of the recipients lie on the 
diagonal, as indicated by the cells with the largest blocks. Over 50 
percent of all the recipients have approximately the same need under both 
models using reported data. However, it is clear from the exhibit that 
more recipients demonstrate less need under the base year model. About 
27.3 percent demonstrated less need (fell below the diagonal) and 10.7 
percent more need (above the diagonal). Some 26 percent of those 
demonstrated little or no need under the prospective model (e.g. below 
$200^ demonstrated need under the base year model. This results from a 
relatively low base year and higher prospective year income and rep. ent 
potential newly eligible recipients. Since students who demonstrate need 
are not guaranteed Campus-Based funds, it is not possible to determine if 
they would receive funds. Almost 5 percent of those who demonstrated 
little or no need under the prospective model demonstrated more than 
$5,000 in need under the base year model (0.61 of the 11.9 percent of 
recipients with $200 or less need using reported prospective data). 
About 9 percent of those who demonstrated need less than $200 under the 
prospective model demonstrated need greater than S200 under the base year 
model . 
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EXHIBrr 3-1. COMPARISON OF CAMPUS-fiASED NEED UNDER PROSPECTIVE AND 
BASE-YEAR MODELS USING REPORTED DATA INDEPENDENT STUDENTS 
(PERCENTAGE BLOCK CHART) 
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Analysis using best data, represented in Exhibit 3-2, indicated that 
the true effects are greater than the apparent effects. (Table B-2 in 
Appendix B presents more data.) This exhibit shows that fewer recipients 
receive about the same awards using best data than using reported data. 
About 5 percentage points fewer recipients, or 54 percent, had 
approximately the same need under both models using best data. Those 
recipients who moved from no or little need under the prospective model 
to demor-trating greater than S200 need under the base year model 
increased to 32 percent (5.86 of the 18.21 percent of recipients with 
$200 or less need using best prospective data). Five percent of those 
demonstrating little or no need under the prospective model demonstrated 
over $5,000 need under the base year model. 

3.2.3 Effects On Error 

Another of the potential effects of shifting from prospective to base 
year income is an increase in the accuracy of the data. Base year data 
are verifiable with IRS or employer records while prospective data are 
not. Table 3-2 presents the need error attributable to income under the 
base year and prospective income models. (Need error is the difference 
in need calculated using reported and best values.) The error rate u..der 
the base year model is less than half (24.7 percent) the error rate for 
the prospective model (56.2 percent). The frequency and amount of 
overestimates of need are lower under the base year income model, about 
20 percentage points and over $400 less respectively, and therefore 
result in a need error that is less than half the prospective model. The 
same is observed with understatements of need. Use of base year income 
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EXHIBIT 3-2. COMPARISON OF CAMPUS-BASED NEED UNDER PROSPECTIVE AND 
BASE. YEAR MODELS USING BEST DATA INDEPENDENT STUDENTS 
(PERCENTAGE BLOCK CHART) 
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TABLE 3-2 

CAMFUS-BASED STUDBIT NEED ERROR ARRIBUTBD 
TO INOQMB FOR INDEPENDQfT STUI»ITS UNDER THE 
PROSPECTIVE AND BASE-TEAR MODELS, 
1985-86 



NO ERROR 
(Within $50) 
Percent 



UNDERSTATQfBIT 



ovHSTATBiarr 



Total Total 
Percent Mean ($ M illi«Mi8) Percent Mean ($ Millions) 



MET MEED BWMR 
Total 
Mean ($ Millions) 



Independent Student Income 
Error Under the 

Prospective Income 43.8 



^ Model 



13.8 



1,100 



62.0 



42.4 1,729 



311.0 



609 



248.9 



Independent Student Income 
Error Under the 

Base Year Income 75.3 
Model 



2.9 



1,598 



18.8 



21.9 1,369 



122.3 



253 



103.5 
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'J 



91 



fcERjC 



results iQ l«ss frequent but larger understatements of need, more than 10 
percentage points less but almost $500 more on average. Base year mean 
net error and total net error are less than half that for the prospective 
model . 

3.3 EFFECTS ON PELL RECIPIENTS 

Reauthorization of the Higher Education Act also prescribes changes 
in the Pell Family Contribution Schedule. These changes will not affect 
as large a number of recipients as are affected by the changes to the 
UM. This is a function of the manr.er in which prospective income is used 
in the Family Contribution Schedule. Dependent Pell applicants supply 
both base year and estimates of prospective year income. If prospective 
income is less than 60 percent of the base year income, the processor 
uses the prospective income estimate to calculate the SAI . This was 
devised to accommodate eases in which income drops between years. 

However, Table 3-3 indicates that this program feature is having 
completely the opposite effect. This table presents the proportion of 
dependent recipients for whom prospective data were actually used 
(reported data) and the proportion for whom it should have been used 
(best ("ata). The proportions using prospective under reported and best 
data are virtually equal (35.6 percent and 33 percent). However, awards 
for about 29 percent (15.7 percent plus 13.1 percent) of the dependent 
recipients were calculated using the wrong data (either base year when 
they should have used prospective^ or the opposite). These data indicate 
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TABLE 3-3 

PERCBMT OF OEPEHDEHT PELL EECIPIENTS FOE 
MHOM PHOSPECTIVE INCOME WAS USED, 
1985-86 



TTRIMG REPORTED !fALUES 



Prospective 
Not Used 



Prospective 
Used 



TOTAL 



Prospective 
Not Used 



51-2 



15-7 



66*9 



US NG 

BEST VALUES 



Prospective 
Used 



TOTAL 



13.1 



64.3 



19-9 



35-6 



33.0 



99-9 
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that while th« progreun is using prospective income for approximately the 
correct number of dependent recipients, the 29 percent error rate 
suggests that they are entirely the wrong group of dependent students. 
This finding clearly indicates that the change in HEA was warranted. 

3.3.1 Effects on Current Pell Recipiento 

The use of base year income for will affect 20 percent of Pell 
recipients, as discussed above. However, as discussed in Section 3.2, 
the change can result in different effects using reported and best data. 
Apparent effects result when analyzing the changes caused by shifting 
from prospective to base year for this 35 percent of the population using 
only reported values. True effects are uncovered by conducting this 
analysis using best values. Although this difference is potentially 
large in some analyses, the effects are small here. Exhibit 3-3 
indicates that 81 percent of Title IV recipients receive the same Pell 
award amount (including $0). Fifteen percent become ineligible, although 
45 percent of these had a positive award of less than $750. No 
recipients received increased awards. 

Analysis of the true effects, portrayed in Exhibit 3-4, indicate only 
minor differences in this analysis, 82 percent receive the same award 
(including $0) and slightly less, 12 percent, became ineligible. A small 
number, just 1 percent, increase their awards. 
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EXHIBIT 3*3. PEU AWADOS USING ALTERNATIVE DEPENDENT INCOME MODELS 
FOR RECIPIENTS WITH PROSPECTIVE INCOME LESS THAN 
60 PERCENT OF BASE YEAR INCOME - REPORTED DATA 
(PERCENTAGE BLOCK CHART) 
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FOR RECIPIENTS WITH PROSPECTIVE INCOME LESS THAN 
eo PERCENT OF BASE YEAR INCOME - BEST DATA 
(PERCENTAGE BLOCK CHART) 
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3.3.2 Effects on Error 



As anticipated, the change from prospective to base year causes a 
precipitous decline in most measures of error. Table 3-4 indicates that 
Peli error drops from 16.1 to 12.2 percent and net error drops $68 
million from $64.2 million to -$4.4 million. However, underawards 
increase from a negligible level -.o nearly 8 percent of cases and over 
$22 million. Thus, while the net error associated with dependent student 
income virtually would be eliminated by the change, underawards would 
increase slightly. 



3.4 CONCLUSIONS 

The analysis of altering the UM and Pell formula from using 
prospective to base year data result in several important conclusions. 
These include: 

Campus-Based Programs 

• All models, reported prospective, reported base year, and 
best base year, result in substantial differences from the 
.distribution of need resulting from the best prospective 
model, although reported prospective was superior to the 
others . 

. Replacement of reported prospective data with reported base 
year data results in frequent apparent changes in need, with 
a pronounced downward shift in need. 

• Replacement of best prospective data with best base year data 
indicates that the true effects are greater with higher 
percentages of recipients changing need and a greater 
downward shift in need. 
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TABLE 3-4 



PELL GRANT AWARD SOKK ATTRIBUTABLE 
TO DEPENDENT STUDENT INCXIMB ERROR UNDER THE 
PROSPECTIVE AND ENTIRE BASE-YEAR MODELS, 
1985-86 



NO ERROR UNDERAW ARDS OVERAWARDS NET HRHOR 

(Within $50) Total Total Total 

Percent Percent Mean ($ Millions) Percent Mean ($ Milliona) Mean ($ Millioog) 



Dependent Student Income 
Error Under the 

Prospective Income 83*9 0.40 400 0.8 15.7 780 65.0 120 64,2 

I Model 

00 

Dependent Student Income 
Error Under the 

Base Year Income 87.8 7.9 531 22.2 4.4 769 17.8 -8.2 -4.4 

Model 



r 

E; 

K • Quality is improved substantially under the base year model 

\ with error less than half that of the prospective model. 

hi 

Peil Program 

I • Use of prospective income for dependent Pell recipients is 

: error-prone and the use of base year income affects less than 

20 percent of the recipients. 

,< 

• The impact of the change is greatest among recipients with 
low awards. Nearly half of those becoming ineligible had 

; awards of less than $750. 

• Pell error attributable to dependent students* income is 
reduced under the base year model from 16 to 12 percent and 
net error drops by $68 million. 

These conclusions suggest that eliminating the use of prospective 
income in the Uniform Methodology and Pell formula as a corrective action 
response will have different effects. Although quality in the 
Campus-Based^ programs will be improved substantially by using base year 
income, the change will cause a pronounced downward shift in need for 
independent students. Analyses conducted indicated no systematic, 
effective means of eliminating or minimizing these need changes. 
However, the reauthorization of the Higher Education Act allows for the 
fact that institutional discretion can be used^ to^-deal with large need 
changes . 



The shift to bai" year income will have a less significant impact on 
recipients in the Pell program. In light of the substantial improvement 
in quality, these effects are likely to be judged acceptable by ED. In 
addition, reductions in Pell awards will increase need for Campus-Based 
and GSL aid, potentially minimizing the impact. 
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ANALYSIS or REDSriHlBD DEF«:.DENCY STATUS 

Beginning in the 1987-88 academic year, a redefined set of data 
elements and rules will be used to determine the dependency status of 
applicants for Title IV aid. While the reasons for the redefinition may 
transcend strictly quality-related issues, because dependency status 
errors have been high, and the change in definition is viewed as a 
corrective action, the probable effects, and distributional consequences, 
of the change should be analyzed in this report. The determination of 
dependency status plays a major role in the calculation of need and 
awards in the Title IV programs. In this chapter, we use data from the 
QC study to simulate the expected effects of the implementation of the 
redefined dependency status rules in these three areas: 

» The distribution of recipients by dependency status under the 
current and redefined regulations 

• The anticipated error rates in dependency status under the 
current and redefined regulations, and 

• The major determinants of dependency status under the 
redefined regulations. 

This analysis allows us to reach conclusions regarding the likely impact 
of the redefined regulations and to identify ways in which the rules 
could be modified to achieve different results. To summarize, our 
ana.ysis revealed the following: 
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. T„ aogreaate, there is no change in the distribution of 

independehts and dependents between the current and redefined 
dependency status regulation. An estimated 14 percent of 
recipients change dependency status under the redefined 
model, however, 7 percent going from independent to dependent 
and 7 percent from dependent to independent. Recipients who 
are 22 or 23 years old are the most likely to change 
dependency statui under the redefined model. 

• The percentage of recipientF reporting as independents who 
should have been dependents is lower under the redefined 
model, but the percentage of recipients reporting as 
dependents who should have been independent is higher. 
Because the redefined model changes the incentives for 
students to misreport, the estimated error rate calculated 
under the redefined model should be treated cautiously. 

• Modifying the redefined model to espand the scope of the 
self-sufficiency criteria greatly reduces the problem of 
recipients who are dependent under the current model becoming 
independent under the redefined model. Unfortunately, the 
modification also causes a number of current independents to 
become dependent under the redefined model. 

The nature of the change in dependency status is such that it is 
classified as a Level II corrective action. This change affects the 
definition, and hence treatment, of a group of data elements, in this 
case, dependency status. 



4 . 1 BACKGRCXJND 

In 1985-86 dependency status was defined according to the responses 
to the following six questions: 

• Did the student live with his/her parents for more than 6 
weeks (42 days) in the preceding year? 

• Will the student, live with his/her parents for more than 6 
weeks (42 days) in the coming year? 
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• Did the parents claim the student as a U.S. tax exemption in 
the preceding year? 

• Will the parents claim the student as a U.S. tax exemption in 
the coming year? 

• Did the student receive more than $750 worth of support from 
his/her parents in the preceding year? 

• Will the student receive more than $750 worth of support from 
his/her parents in the coming year? 



A -yes- response to any of the six questions means that the person is 
treated as a dependent in applying for Title IV aid. The problems with 
the current definition of dependency status are fairly obvious. Three of 
the questions refer to activities that will not take place until the 
coming year. Consequently, applicants must project their circumstances 
for a fut. J time period, which leads to inaccuracies. In addition, the 
use of projected data makes these items unverifiable at the time of 
application. Questions concerning living with, or receiving support 
fro^. parents may seerr. arbitrary and, if they involve small amounts of 
time or money spread throughout the year, may be difficult to reconstruct 
even for a prior time period. 

AH of the QC studies have identified dependency status as a major 
source of dollar error in the delivery of Federal student financial 
assistance. In Findings it was shown that students improperly receiving 
aid as independents had a significant impact on program-wide error rates 
in both the Pell and Campus-Based programs. Table 4-1 presents 
information concerning which of the six elements comprising the current 
dependency status definition were most likely to be misreported by 
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TABLE 4-1 

PERCENTAGE OF INDEPENDENT TO DEPENDENT SWITCHERS 
WITH DISCREPANCIES IN THE ITEMS DEFINING DEPENDENCY STATUS, 
ALL TITLE IV PROGRAMS, 1985-86 

PERCENTAGE OF INDEPENDENT 
TO DEPENDENT SWITCHERS 
IX?34 WITH DISCREPANCIES 

Lived with in 1984 43.7 
Lived with in 1985 52.8 

Claimed in 1984 28.0 
Claimed in 1935 23.6 

Support in 1984 51.5 
Support in 1985 49.0 




recipients claiming to be independents who should have been dependents. 
The table contains information at the item level across all programs. 
Not surprisingly, the questions, about living with or receiving support 
from parents were most often misreported. The questions asking whether 
the parents did or will take the student as a U,S. tax exemption are more 
easily answered and verifiable, and therefore were less often misreported. 

In redefining dependency status, an attempt was made to avoid the 
pitfalls of the current definition. The questions concerning living 
arrangements or financial support from parents were eliminated, Th-i data 
elements included in the definition were chosen in an effort to make 
dependency status understandable and verifiable. The objective was to 
make the new'def inition simpler and more accurate than the current one. 

Under the new system, dependency status is redefined in the following 
manner, A student is considered independent if he/she meets any of four 
conditions that we call Level 1 criteria. The Level 1 criteria include 
the following: 

• Is 24 years of age or older as of December 31 of the award 
year 

• Veteran of the U,S. Armed Forces 

• Orphan or ward of the court, or 

• Having legal dependents other than a spouse. 



Failing to meet any of the Level 1 criteria, a student is considered 
independent if he/she meets both of the Level 2 criteria as follows: 

• Being married and/or a graduate student during the academic 
year/ and 

• Not expecting to have his/her parents claim them as a U.S. 
tax exemption in the coming year. 

Uninarried undergraduate students are considered independent if they meet 
the Level 3 criteria which are the following: 

• Parents did not claim them js U.S. tax exemptions in either 
of the two preceding years, and 

• They received $4,000 or more in total income and benefits in 
both of the two preceding years. {Technical amendments to 
the Higher Education Act changed this criterion after we 
analyzed redefined dependency status. This criterion now 
includes student financial assistance in the $4,000 in annual 
total resources, but assistance from parents is not 
included. ) 

In the remainder of the chapter we analyze the likely impact of this new 
dependency status definition on the distribution of recipients in the 
Title IV programs . 



4 . 2 KETHODOLOGY 

The first step in the analysis was to use the QC data base to 
simulate the impact of redefined dependency status with both reported and 
best data. There are two data elements in the redefined formula for 
which there were no data available in the QC data base. No data was 
collected in the study on circumstances occurring 2 years prior to the 
academic year. Consequently, we do not know if an unn^arried 
undergraduate student was claimed by his parents as a U.S. tax 



exemption or received $4,000 or more in total income end benefits in the 
year 2 years prior to the academic year (i.e., 1983). both of which are 
part of redefined dependency status. The impact of the unavailability of 
these data elements requires classifying recipients as independent under 
the redefined rules when they may have been claimed as a U.S. tax 
exemption by their parents, or may not have received $4,000 in income and 
benefits in 1983. Of course, to be classified as independent, these 
recipients had to have not been claimed as a U.S. tax exemption by their 
parents and received $4,000 or more in income and benefits in the year 
preceding the present academic year (1984), for which data was 
available. Therefore, it is not unreasonable to assume that the 
conditions occurring in 1984 also occurred in 1983, so that the effect of 
the missing data elements should be minor. 

As in Chapters 3 and 6 the simulations produced four models: 

• Current dependency status using reported data 

• Current dependency status using best data 

• Redefined dependency status using reported data, and 

• Redefined dependency status using best data. 

We determined the distribution of independents and dependents that 
occurred under each of the four models. Unlike analyses in Chapters 3 
and 6, however, we cannot determine awards for each of the models. The 
problem arises for recipients who are defined as independents using both 
reported and best data under the current system who become dependents 
under the redefined system. For these students we have no information on 
their parent's circumstances and so cannot compute awards for them as 
dependent recipients. 
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Instead, we had to limit the analysis to assessing changes in the 
distribution of dependents and independents under the four models. 
Because the analysis can only be conducted at the item and not the award 
level, it does not make sense to present the results by Title IV 
program. Rather, results are aggregated across programs so that a single 
set of results can be presented. These results are presented in the next 
section. 

4.3 EESULTS AND CONCLUSIONS 

Several different types of analyses were conducted. Comparing the 
distribution obtained under the redefined rules to the current 
distribution- indicates the overall impact of the redefinition. Crossing 
these comparisons with relevant student characteristics indicates the 
impact of the redefinition on subpopulations of interest. Contrasting 
repr .ed to best data for both the current and redefined models produces 
error rates under each system. Comparing error rates across the two 
systems indicates the extent to which validity is affected by the 
redefinition of dependency stex.us. Finally, measuring the impact of the 
specific data elements comprising the redefined rules on the 
d^ftermi nation of dependency status allows us to identify ways that the 
redefined rules could be changed to achieve different results. 



4.3.1 Impact of the KedeCinition of Dependency Status 



In order to assess the impact of the new definition along several 
dimensions, we make four comparisons of the models and then supplement 
these comparisons by looking at subpopulations of recipients. The four 
basic comparisons we examine are the current model using reported data 
vs. the redefined model with reported data, the current model with 
reported data vs. the redefined model with best data, the current model 
with best data vs. the redefincjd model with reported data, a>»d finally 
the current model with best data vs. the redefined model with best data. 
These comparisons are discussed in the following paragraphs. 

Table 4-2 presents the distribution of dependents and independents 
under the current and redefined system using both reported and best 
data. Comparing the current reported model with the redefined reported 
model simulates the expected redistribution that will occur when the 
redefined rules are implemented. The total percentage of independents 
(41.9) and dependents (58.1) remainr constant under the redefined 
reported model. This does not mean tl the redefinition does not have 
an impact. An estimated 14 percent of recipients change dependency 
status under the redefined system, 7 percent going from independent to 
dependent and 7 percent from dependent to independent. While these 
changes counteract each other in the aggregate, their magnitude indicates 
that on an individual basis there is a great deal of shifting of 
dependency status under the redefined model. 
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CURRENT 



Reported 



IMPACT or REDEFINED DEPEKDEMCY STATUS. 
ALL TITLE IV PROGRAMS, 1985-86 

REDEFINED 



Independent 
Dependent 



Reported 

Independent Dependent 



34.9 
7.0 



7.0 
51.1 



Best 

Independent Dependent 



35.4 
7.7 



6.5 
50.4 



Best 



Independent 
Dependent 



33.7 
8.3 



6.3 
31.8 



34.5 
8.6 



5.4 
51.5 
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On* goal of the redefinition of dependency status was to incorporate 
data elements that could be verified more easily. If verifying component 
data elements became part of the implementation of the redefined 
dependency status model, then using best rather than reported data might 
better simulate the expected impact of the redefined model. Comparing 
the current reported model to the redefined best model reveals similar 
results as the comparison to the redefined reported model. Using best 
data, the percentage of recipients going from independent under the 
current model to dependent under the redefined model (6.5 percent) 
declines slight.ly hile the percentage going from dependent to 
independent (7.7 percent) increases slightly, compared to the results 
obtained using reported data. 

In Chapters 3 and 6, comparisons of the redefined reported model with the 
current best model are used to assess the equity of the new definition. 
This assumes that the current best model is the optimal one and that the 
redefinition is only attempting to get closer to the optimal than the 
current reported model by removing the error inherent in the current 
model. In the case of dependency status, the assumption that the current 
best model is optimal may not be entirely valid. Quality issues 
certainly were a major impetus for the redefinition. However, there was 
also a feeling that the current definition did not adequately capture the 
phenomenon of dependence and so needed to be redefined. The extent to 
which the redefinition was predicated on conceptual factors, rather than 
quality related factors, determines the validity of comparisons to t*ie 
current best model as optimal. 



The data in the table reveal that the redefined models do not 
approach the current best model. Using reported data, 14.6 percent of 
recipients change status from the current best model to the redefined 
model, with 8.3 percent going from dependent to independent and 6.3 
percent going from dependent to independent. Using best data the results 
do not change significantly. The redefined best model differs from the 
current best model in 14 p-rcent of cases, with 5.4 percent going from 
independent to dependent and 8.6 percent dependent to independent. 

Under both redefined models, the impact of the change compared to the 
current best model is far greater than the dependency status error rates 
occurring under the current model in Findings . The importance of this 
result depen-ds on whether the redefinition was solely intended to remove 
the error inherent in the current model or was based on a different 
philosophy of dependence. If quality issues predominated, then the 
redefined model does not appear to be successful given its large 
deviation (percentage of switchers) from the current model under ideal 
circumstances (i.e., us ing best data) . 

Exhibit 4-1 demonstrates the impact of redefined dependency status, 
using reported data, according to the student's age. The vast majority 
(92.4 percent) of recipients 18 or under are dependent under both 
models. Of the 6.3 percent that are independent under the current model, 
two-thirds become dependent under the redefined model. Similar results 
hold for recipients between 19 and 21, with 82 percent remaining 
dependent, but almost half of those independent under the current model 
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becoming dependent. Recipients who are 22 or 23 are the most likely to 
switch dependency status with 11 percent going from dependent to 
independent and 13.7 percent going from independent to dependent. By 
definition, all recipients 24 or older are independent under the 
redefined model. However, 12.5 percent of these recipients are dependent 
under the current model. 



4.3.2 Comparison of Error Rates Under the Two Models 



As stated above, a major impetus for redefining dependency status was 
the misreporting problem occurring under the current definition. To 
assess whether the redefined model is likely to improve validity we 
compared the error rate found under the current model to the error rate 
expected under the redefined model. The results of this comparison are 
presented tn Table 4-3. 

The table indicates that under the redefined model the rate at which 
recipients report being independent, when in fact they should be 
classified as dependent, is reduced by one-half. The implementation of 
the redefined model, therefore, should significantly reduce the problem 
of independent to dependent switchers identified in this and previous QC 
studies. As shown in Findings , the impact on award error of switching 
from independent to dependent status is much greater than switching from 
dependent to independent. Under the redefined model the percentage of 
recipients who report being dependent who should be classified as 
independent increases substantially . 
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COMPABISOH OF ERROR RATES BETWEEN THE CURRREHT 
AMD REDEFINED DEPENDENCY STATUS MODELS, 
ALL TITLE IV PROGRAMS, 1985-85 



ERBOR RATE UNDER CURRENT DEFINITION 



Reported 





Independent 


Dependent 


Total 


Independent 


39.3 


2.6 


41.9 


Dependent 
Total 


C.6 


57.5 


58.1 


39.9 


60.1 


100.0 



RgROR RATE UNDER REDEFINED DEFINITION 



Reported 



Independent 

Dependent 

Tota: 



Independent 

40.6 
2.5 
43.1 



-BEST- 



Dependent 


Total 


1.3 


41.9 


55.6 


58.1 


56.9 


100.0 
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Even given the diminished impact, on award error, the increase in the 
rate of dependent to independent switchers under the redefined model is 
troubling. The reason for this increase probably has to do with the 
misreporting of student's income and benefits since there was little or 
no misreporting identified in any of the other elements that comprise 
dependency status under the redefined model. Unmarried undergraduates 
may underreport their own incor..e and benefits sc that, using reported 
data, they fell below the $4,000 limit and are considered dependent under 
the redefined model. Substituting best values, the amount of income and 
benefits received by these unmarried undergraduates may be revealed to be 
$4,000 or greater causing them to be classified as independent under the 
refined model. 

The above explanation assumes that the incentives for misreporting 
remain the same under the current and redefined models. In Findings, it 
vas demonstrated that the majority of recipients misreportiny, do so in a 
manner that increases their award. Under the current system, there are 
incentives for recipients to underreport their own income and benefits 
because SAI and need are inversely related to income. Under the 
redefined model, the incentives for misreporting student's income are 
altered. Reporting an income of $4,000 or more may prove beneficial if 
the associated increase in the student's contribution is more thai offset 
by the savings associated with being treated as an independent. 
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Given the relationship between misreportiug and the peculiarities of 
the formula being used to determine benefits, it is difficult to estimate 
an er-ror rate for the redefined dependency status model. In the analysis 
of error rates presented in Chapters 3 and 6, the incentives for 
misreporting remain the sa.e under the current and redefined models. 
Therefore, it is appropriate to assume that the nature of misreporting 
also remains the sax.e under the redefined model,, Since the incentives 
for misreporting change with the redefined dependency status model, it 
may no longer be appropriate to assume that the nature of misreporting 
found under tho current model also pertains under the redefined model. 

4.3.3 Analysis of the Determinants of Redefined Dependency Status 

AS discussed in Section 4.3.1, determining whether the redefinition 
of dependency status is an effective strategy depends on what the 
objectives for changing the definition are. Given the problems under the 
current system of students receiving aid as independents who should be 
dependents, one reasonable objective of the redefined model might be to 
ensure that these independents are classified as dependents while 
minimizing the number of previously defined dependents becoming 
independents. The data presented previously indicate that if this is the 
objective of the redefined model, it has to be considered unsuccessful. 
The number of dependents under the current model that become independents 
under the redefined model is equal to those changing from independents to 
dependents. The question then becomes: Can the redefined model be 
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altered so that recipients classified as dependent under the current 
model do not become independent under the redefined model? In the 
remainder of this section we address this question by looking at the 
determinants of dependency status using the redefined model. 

The redefined regulations for determining dependency status can be 
thought of as having three hierarchical levels. MeeUng the conditions 
for being independent at any level in the hierarchy results in the 
applicant being treated as an independent in the determination of need 
and/or awards in the Title IV programs. The determinants of independent 
status at each level are listed on Table 4-4. 

Because the levels are hierarchical in na^ -re, recipients meeting the 
criteria for independent status at a higher level do not also need to 
meet them at the lower levels. For example, any recipiert 24 or older 
has no other conditions for being considered independent. To understand 
the critical determinants of dependency status under the redefined model, 
it is instructive to analyze the levels as concurrent rather than 
hierarchical. In Table 4-4 the percentage of cases classified as 
independent under the redefined model meeting .he criteria at each level 
is presented regardless of whether they met the criteria at a higher 
level. Married or graduate students are also checked against the 
unmarried undergraduate criteria which is an indication of 
self-sufficiency. Table 1-4 presents the breakdown by level depending on 
whether the recipient is independent or dependent under tho current 
definition of dependency status. 
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TABLE 4-4 



DETERMINANTS OF INDEPkSIDENT STATUS UNDER 
THE REDEFINED DEFINITION, 1985-86 



GJRRENT STATUS 
Independent Dependent 



Level 1* 

Level 1 only 
Levels 1 and 2 
Levels 1 and 3 
Levels 1,2, and 3 



86.2 



66.8 



21.6 
11.7 
31.0 
35.7 
100.0 



65.1 
12.1 
21.5 
1.3 
100.0 



Level 2* 

Level 2 only 
Levels 1 and 2 
Levels 2 and 3 
Levels 1, 2 , and 3 



45.6 



16.7 



4.5 
22.1 
5.8 
67.6 
100.0 



46.1 
48.4 
0.0 
5.5 
100.0 



Level 3* 

Level 3 only 
Levels 1 and 3 
Levels 2 and 3 
Levels 1,2, and 3 



69.2 



40.8 



13.1 
38.6 
3.8 
44.5 
100.0 



62.7 
35.1 
0.0 
2.2 

100.0 



* Level 1: 

Is 24 years of age or older as of December 31 of the award year. 

Veteran of the U.S. Armed Forces, 

Orphan or ward of the court/ or 

Having legal dependents other than a spouse. 

* Level 2: Married or graduate students 

Not expecting to have their parents claim them as a U.S. tax 
exemption in the coming year. 

* Level 3: Unmarried undergraduates 

Parents did not claim them as U.S. tax exemptions in either of the 
two preceding years, and 

They received $4,000 or more in total income and benefits in both 
of the two preceding years. 

For exantple: 86.2 percent of the independent students und#»r the current 
definition met the Level 1 criteria under the new 
definition. Of these, 21.6 met only Level 1, 11.7 Levels 
1 and 2, 31.0 Levels 1 and 3, and 35.7 met Levels 1, 2, 
and 3. 
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For both grot.ps, the Level 1 criteria is the one most often met. The 
majority of recipients qualifying as i. dependent at Level 1 did so based 
on being 24 or older. Recipients who are independent under the current 
definition are more likely to meet the criteria at each level than those 
who are currently dependent. In addition, there is much less overlap 
among levels for current dependents. For example, 65.1 percent of 
current dependents only met the Level 1 criteria compared to 21.6 percent 
of current independents. 

The results indicate that a large number of recipients classified as 
independent under the redefined model do not meet the Level 3 
self-sufficiency criteria (not claimed as a U.S. tax exemption by their 
parents and receiving $4,000 or more in income and benefits in the 
preceding 2 years). This is particularly true of current dependents, 
where almost 60 percent fail to meet the Level 3 criteria. Imposing the 
Level 3 self-sufficiency requirements on all students, regardless of 
whether they qualify at a higher level, should therefore help eliminate 
much (60 percent) of the problem with recipients ^oing from dependent 
under the current model to independent under the redefined model. 
Imposition of the Level 3 criteria on all students also causes a 
significant (30 percent) number of independents under the current model 
to become dependent. 
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If economic self-sufficiency is to be the critical criterion for 
independent status, then students, regardless of age, should be able to 
demonstrate how they support themselves or be considered dependent on 
parental support. Therefore, we recommend that one criterion for 
independent status be the student's ability to demonstrate 
self-sufficiency. 
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INSTITUTIOKAL QUALITY CONTROL 
PROCEDUBES AND VALIDATIOH^ 

This chapter presents our analyses of institutional quality control 
procedures and validation. The analyses presented in this chapter build 
upon those presented in Findings , where preliminary analyses suggested 
that institutional quality control procedures and validation were 
associated with lower rates of institutional and student error, 
respectively, in the Title IV ^-rograms. This chapter will further 
analjze .uality control procedures and validation to determine the extent 
to which each is related to student and institutional error, when 
controlling for other factors that may have influenced the results 
presented in Findings • 

Restructuring validation and implementing or focusing corrective 
actions on an institutional quality control approach are Level III 
corrective actions. These types of corrective actions are designed to 
address residual error after Level I and II corrective actions. By 
changing or restructuring validation procedures or focusing 
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Throughout this chapter we use the term "validation" to refer to 
activities in the Pell program that are associated with 
confirming the value of student-supplied data. In addition, we 
also use the term to refer to optional activities of 
institutions to verify the data outside the scope of the Pell 
Grant program. We use the term validation to avoid confusion 
since the activities included in this chapter occurred in the 
1985-86 academic year, prior to recent regulations that 
formulated an integrated system. 



institutional quality control ED is se«king to remove any disparity 
between lines of authority, control, and methods of accountability. 



The results in this chapter are different from those in Findinqs and 
could produce different outcomes becausfe the analyses in this chapter 
control for effects among the variables while the analyses in Findinqs 
did not. Variables that appeared significant in the analyses in Findings 
may not be significant in these analyses. 

In order to control for these other factors, the analyses in this 
chapter use multivariate models involving regression equations. In 
general using regression analysis allows one to isolate the effects of a 
given factor or variable on an outcome measure (i.e., the dependent 
variable) when controlling for various other independent variables. In 
the analyses in this chapter, the independent variables in each model are 
tested for their relationship with the existence of error. Testing 
variables in regression models allows us to estimate the association 
between each of the independent variables and the likelihood of error. 
The analyses we performed for this report indicate the following: 

Institutional Quality Control Procedures 



« For the moc^ part, the analyses conducted in this report 
confirn> those that we presented in Findinqs. Institutional 
quality control procedures were generally associated with 
lower rates of institutional error in the Pell and 
Campus-Based programs. In many cases, however, the lower 
rates of error were not statistically significant at the 10, 
5, or 1 percent level. 
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• Sampling-based QC procedures continued to be associated with 
lower rates of error more often than other QC procedurss. 
The QC category other/sampling was significant at the 5 
percent level in four of the five models, while the category 
automated/sample was significant at the 1 percent level in 
two of the five models. 

• Higher levels of either professional or clerical/data entry 
staff and higher levels of automation were, for the most 
part, not significantly associated with lower rates of 
institutional error. However, these variables could be 
related to the number of recipients, or other variaoles, 
which were significantly associated with lower error rates. 

• These findings support continued development of the 
Institutional Quality Control Pilot Project and similar 
activities. In addition to the Pilot, materials and 
information concerning quality control procedures could be 
developed as technical assistance materials for institutions 
not participating in the pilot. 



Validation 



• Confirming the analyses in Findings , the analyses in this 
chapter show that Pell validated cases have a higher 
probability of having a student error removed than not 
validated cases. 

• Institutional validated cases also have a higher probability 
of having errors removed than not validated cases. This 
difference, however, is not significant. 

• Validation-related procedures that could be used in designing 
corrective actions were not significant in explaining 
differences in error removed through validation. 



5.1 IMSTITUTIOWAL C^ALITY CCmROL PROCEDUKES 



Initial analyses performed in Findings pointed to the fact that 
students attending institaions that employed quality control procedures 
had lower rates of institutional error in the Pell and Campus-Based 
programs than students attending institutions that did not employ quality 
control procedures. In addition, the Findings results suggested that 



certain QC procedures worked better than others. However, the Findinas 
analyses did not control for any other variables that might have been 
related to an institution's use of quality control procedures. 

The following sections of this chapter will present the outcome of 
further examination of institutional quality control procedures. We have 
conducted our analyses to control for other institutional 
characteristics. This allows us to determine the association between 
institutional quality control procedures and institutional error rates 
more accurately than if we did not control for other characteristics. 
When presenting our analyses we continue to use the categories we used in 
Findings to classify institutions according to the types of QC procedures 
they employed. A complete description of these categories can be found 
in Chapter 7 (Section 7.2.1) of Findings . 

5.1.1 Sumnary from Findings 

As mentioned above, in the Findings report we found that students 
attending institutions that had a set of quality control procedures in 
place had lower rates of institutional error than students attending 
institutions with little or no quality control procedures. The findings 
analyses found this relationship to be present in the P^U and 
campus-Based programs. By contrast, q-.ality control procedures weri not 
associated with lower rates of institutional error in the CSL program. 
Table 5-1 summarizes the Findings results for both the Pell and 
Campus-Based programs. 



TABLE 5-1 

SOMAKT OP FIMDUCS RESULTS FOR QUALITT GGMnnL PROCBUaB, 1985-86 



PC CMrBQOBY mStlTUTIOMM. PgLL KBROR RATE (t) 

No Error 

(Within $50) Utoaeraward Qgarawtrd 

Little or Ko 



Qudixty uontroi 




12.5 


26.0 


Mixed 


66.1 


11.7 


22.2 


Automated 


68.4 


8.6 


23.1 


Sample 


70.2 


14.4 


15.5 


Automated/Sample 


76.8 


8.0 


15.2 


Manuai/Other 


62.6 


16.2 


21.2 


Manual or 
Other/Sample 


81.8 


10.4 


7.8 



IMSTITUTIOMM. CMITOS-BASH) MBD EBBOR RME (t) 
No Error 

(Within $50) Utadar«tat—ent» Owaratat—wif 



JO • ^ 


20.4 


26.3 


73.5 


14.8 


11.7 


61.9 


20.4 


17.7 


67.8 


12.0 


11.7 


76.3 


14.4 


17.9 


61.6 


15.4 


23.0 


73.4 


8.7 


17.9 



1? 



1 



Pell Graat Proaraa 

S.udents attending institutions that made use of institutional 
quality control procedures had lower rates of institutional error. In 
particular, the analysis pointed to the fact that certain types of 
quality control procedures might be associated wix.h lower levels of 
institutional error than other types of quality control procedures. The 
analysis in Finding s suggested that sampling was associated with the 
lowest rates of institutional error particularly when sampling was used 
in conjunction with other quality control procedures (e.g.. automated 
quality control checks, manual quality control checks, or other auxiliary 
quality control procedures). Students attending institutions using 
sampling and either manual quality control checks or auxiliary quality 
control procedures (e.g., using auditors, using consultants, checking 
other offices, or interviewing students) had institutional Pell error 
19.2 percent of the time, students attending institutions using sampling 
in conjunction with automated quality control checks had institutional 
Pell errors 23.2 percent of the time, and 29.8 percent of the recipients 
attending institutions using only sampling had institutional Peli 
errors. In contrast, 38.5 percent of recipients attending institutions 
with little or no quality control had institutional Pell errors. 

Campus -Based Programs 

The analyses in Findings concerning the association of institutional 
Campus-Based errors with quality control procedures found results similar 
to those in the Pell program. Students attending institutions using 
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sampling in conjunction with other q-'^.lcy control procedures, as well as 
students attending institutions using a mixture of quality control 
nrocedurec had lower rates of institutional error. Students at schools 
using sampling in conjunction with automated quality control checks had 
institutional Campus-Based need errors 23.7 percent of the time, students 
attending institutions using sampling in conjunction with either manual 
quality control checks or other auxiliary quality control procedures had 
institutional errors 26.6 percent of the time, while 26.5 percent of the 
recipients attending institutions using a mixture of quality control 
procedures <with no individual type of procedure predominating) had 
institutional Campus-Based need errors. Students attending institutions 
with little or no quality control had institutional errors 46.8 percent 
of the time. 

5.1.2 Methodology and Analytic Approach 

While the differences we identified in Findings for Pell and 
campus-Based were statistically significant, the analysis could not 
determine if the differences in error rates occurred due to QC procedures 
or due to other related institutional characteristics. Because it is 
possible that schools employing sampling or other QC procedures tend to 
have other common characteristics (e.g., type and control, number of 
recipients), the lower error rate could have been the result of these 
characteristics. Therefore, the Findings analyses may have masked the 
effects of other characteristics. For this reason the analyses presented 
in this report have been structured to indicate more effectively the 
association of QC procedures and institutional Pell error. 
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To identify the critical determinant? of error, we must estimate the 
relationship between the individual explanatory factors and the existence 
of error. In order to accomplish this, the impact of a given explanatory 
factor must be estimated, controlling for the effects of other relevant 
factors. In particular, we want to estimate the impact of quality 
control procedures on the likelihood of error, while controlling for 
other institutional characteristics. Because it is necessary to control 
for other factors, we needed to use multivariate techniques to estimate 
the models. The bivariate analyses presented in Findings , while not 
appropriate for estimating the models, were useful in helping to specify 
the multivariate models that we tested. 

Regression analysis is generally considered one of the best 
statistical techniques for hypothesis testing in a multivariate 
framework. Therefore, it is appropriate where we have prior hypotheses 
concerning the relationship between the dependent and independent 
variables. As stated before, the outcome measure used in the equations 
was the probability of an error occurring. The dependent variable is, 
therefore, dichotomous. with a one coded if an error of over $50 was 
present and a zero otherwise. The use of a continuous dependent 
variable, (i.e.. the level of error) is not proper in this situation 
because the large number of cases without error would tend to distort the 
regression results. Also, the level of error is not of particular 
interest since it depends on characteristics such as unmet need that are 
beyond the control of the institution. 



The Ordinary Least Squares (OLS) regression procedure contains 
several estimation problems when the dependent variable is dichotomous. 
These problems include a heteroskedastic error term and the possibility 
of predicting probabilities of over 100 percent or probabilities which 
are negative. Therefore, in order to estimate the models with a 
dichotomous dependent variable we used the logistic multiple regression 
procedure, which overcomes the problems associated with the OLS 
procedure, relating the occurrence of errors as dependent variables to 
the explanatory variables. 

With a larger saniple size it would have been possible to estimate 
models for each of the critical errors identified in Findings . 
Unfortunately, for many of these errors there were not sufficient numbers 
o* cases with error to permit estimation. For example, overall 
institutional error could not be broken into its component parts because 
errors in these components occurred too infrequently. The need to use 
more aggregated error measures as dependent variables may mean that 
explanatory factors which were only related to certain specific errors 
may not have been uncovered and identified as critical determinants of 
error. In addition, it means that no statements regarding the 
determinants of error could be made for several of the more specific 
errors many of which were discussed in Chapter 2, 

The following is a list of dependent variables for w!iich we estimated 
models : 
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• Pell: 



Institutional overawards 
Institutional underawards 

• Campus-Based: 

Institutional overstatements of need 

Institutional understatements of need 

Institutional payment error (awards in excess of need) 

For each of the models we used the same independent or explanatory 
variables. These independent variables included those that imply an 
approach to quality control and can be varied (e.g., types of QC 
procedures used, use of automation, number of staff FTE's) as well as 
variables that are control variables and cannot be altered (e.g., type 
and control of institution, number of recipients). This distinction is 
important to make since independent variables that can be altered have 
particular relevance for corrective actions. The explanatory variables 
we used in the five models were the following: 



• Institution type and control 

• Academic calendar 

• Level of automation 

• Number of clerical or data entry staff PTE's 

• Number of professional staff PTE's 

• Number of program recipients, and 

• Type of QC procedures used. 



We used these explanatory variables to estimate the effects on the 
likelihood of an institutional error. Because QC procedures, level of 
automation, and number of clerical/data entry and professional staff 
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PTE'S were associated with lower rates of institutional error in 

Findings , we hypothesised that, they would continue to be associated in 

the multivariate models. Cases that had errors of $50 or less were 
treated as if they were not in error. 

5.1.3 Results and Conclusions 

Tables 5-2 through 5-6 (starting on page 5-15) present the results of 
the logistic regression models discussed in the previous section. These 
tables show each of the explanatory variables. their observed 
relationship or effect on the likelihood of an error, and the level of 
significance of the variable. Because six of the seven explanatory 
variables were discrete, it was necessary to omit one value for each of 
these variables from the regression. It is these omitted characteristics 
that comprise the intercept of the regression against which all of the 
other values of the variables are then compared. Thus, in Tables 5-2 
through 5-6, the effect of alternative values of the discrete variables 
on the likelihood of error are stated relative to the omitted values. 
The significance levels of the estimated relationships are stated at 
levels of 1. 5. and 10 percent, or as not significant. These values 
state the probability that the observed relationships of the explanatory 
variables and the liki lihood of error could occur if in fact there is no 
relationship at all. (For example. Table 5-2 shows that students at 
4-year private institutions were less likely to have an I :istitutional 
Pell overpayment than students at 2-year institutions when controlling 
for the other factors in che model. There is a 1 percent chance that 
this difference might not exist for the population, even though we 
estimated .^e relationship from the sample.) 
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We must state our findings in terms of a level of significance 
because the sample will not be exactly representative of the population. 
We have stated our findings in significance levels of 1, 5, and 10 
percent as these are standard values of accepted levels of significance 
for social scientific studies. Beta coefficients are not presented 
because in logistic regression they represent the change in the logarithm 
of the probability of error associated with a one unit change in the 
independent variable and, hence, are not meaningful by themselves. 

As a further means of explaining the findings of the regression 
models given the lack of meaningful coefficients, we developed six 
different profiles of institutions and presented the expected probability 
of error fo'r each of the models for the six profiles. These profiles 
were designed to be descriptive of certain types of institutions. The 
institutional error rates presenced lOr the protiles were calculated from 
the regression results and not observed. Using a standard formula, we 
translated the regression coefficients into probabilities of error and 
thus the effect of each variable is determined controlling for all other 
variables rather than jointly. Therefore, the imputed error rates should 
not be interpreted as what we would observe, but rather as estimates 
based on the elationships among the variables we examined. 

The profiles we developed for explaining the regression results are 
the following: 



• Profile 1 - 4-year private institution, semester-based 
academic calendar, high level of automation, 
moderate number of program recipients, 5 or more 
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professional staff PTE's, 4 or more clerical or 

data entry stair rxc s, uac ^ 

procedures in conjunction with either manual QC 
checks or other auxiliary QC procedures. (We 
will refer to schools meeting Profile 1 criteria 
as *'high automation, manual or other/sampling QC" 
schools. ) 

♦ Profile 2 - 4-year public institution, semester-based 
academic calendar, high level of automation, 
large number of program recipients, 5 or more 
professional staff PTE's, 4 or more clerical or 
data entry staff PTE's, use of sampling in 
conjunction with automated QC checks. (We will 
refer to Profile 2 schools are *'hign automation, 
sampling/automated QC**.) 



• Profile 3 



Profile 4 



4 -year private institution, non semester-based 
academic calendar, low level of automation, small 
number of program recipients, less than 5 
professional staff PTE*s, less than 4 clerical or 
data entry staff PTE s, little or no use of QC 
procedures. (Profile 3 schools will be called 
'*low automation, little or no QC**.) 

4-year institution (private or public), non 
semester-based academic calendar, low level of 
automation, less than 5 professional staff PTE*s, 
less than 4 clerical or data entry staff PTE's, 
low number or program recipients, use of either 
manual QC checks or other auxiliary QC 
procedures. (We wili refer to Profile 4 schools 
as "low automation, manual/other QC" schools.) 



• Profile 5 



• Profile 6 



proprietary institution, non semester-based 
academic calendar, high level of automation, less 
than 5 professional staff PTE*s, less than 4 
clerical or data entry staff PTE's, moderate 
number of recipients, use of a mixture of QC 
procedures. (Profile 5 schools will be called 
"high automation, mixed QC**.) 

2 -year institution, semester -based academic 
calendar, high level of automation, large number 
of recipients, 5 or more professional staff 
PTE's, 4 or more clerical or data entry staff 
PTE'Sv use of automated QC checks. (Profile 6 
schools will be called "high automation, 
automated QC" . ) 
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While these profiles are mutually exclusive, they are not totally 
exhaustive. The profiles are presented to describe some typical 
institutions and do not display all institutions included in the study. 



Pell Logistic Begrcssion Models 

As shown in Table 5-2, a number of the explanatory variables (e.g., 
staff FTE's, automation) that were significantly related to decreased 
error rates in the Findings analyses are not significant in the 
multivariate analyses. These differences occurred because the analyses 
contained in this report control for other variables while the analyses 
in Findings did not. All QC categories had lower rates of error than 
little or no QC, yet only the mixed, sampling, and other/sampling 
categories are significant. High levels of automation actually increased 
the probability of an institutional Pell overpayment relative to low 
levels of automation although the difference is not significant. 

For institutional Pell underpayments. Table 5-3 shows that no QC 
category is significantly different than little or no QC procedures in 
their association with the likelihood of an institutional Pell 
underaward. All QC categories except for sampling, while not 
significant, did show decreases in the probability of error. Contrary to 
what we hypothesized, a high level of automation is significantly 
associated with the increased probability of institutional Pell 
underpayments compared to a low level of automation. 
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TABLE 5*2 



WGISTIC RBGRESSKH BSTIHATKS OF PtOBA3ILm Of PBLL OVBtPAHaaiTS , 



SOURCE 



BPFSCT ON 
PROBABILm 
OF ERROR 



SIGMIFICAlfCB 
LBVBL 



* 2-'Year (Private and Public) 
4-'year Private 

4-Year Public 
Proprietary 

* Non-Semester 
Semester 

* Low Level of Automation 
High Level of Automation 

* Less Than 5 Professional 
Staff FTE*s 

5 or More Professional 
Staff FTE's 

* Less Than 4 Clerical or Data 
Entry Staff iXE's 

4 or More Clerical or Data 
Entry Staff FTE's 

Number of Pell Recipients 

* Little or No QC 
Mixed QC Procedures 
Automated QC Procedures 
Sampling QC Procedures 
Automated/Sampling QC 
Procedures 

Manual /Other QC 
Procedures 
Other/Sampling QC 
Procedures 



N/A 

Decrease 
Decrease 
Decrease 

N/A 

Decrease 
N/A 

Increase 



N/A 

Increase 
N/A 

Increase 
Decrease 
N/A 

Decrease 
Decrease 
Decrease 

Decrease 

Decrease 

Decrease 



N/A 

1 percent 
5 percent 
5 percent 

N/A 

1 percent 
N/A 

Not significant 



N/A 

Not significant 
N/A 

Not significant 
Not significant 
N/A 

10 percent 

Not significant 

1 percent 

Not significant 

Not significant 

1 percent 



*N/A indicates that these characteristics were omitted from the 
regression and are captured in the intercept. 



TABLE 5-^3 

LOGISTIC MGRKSSIOH ESTIMATES Of PEOBABILITI OF PELL OMDERP AYMEHTS , 

198S-86 



SOOTCE 



EFFECT cm 
PSOEABILITT 
OF EKtOB 



8IGHIFICAHCE 
LEVEL 



* 2-Year (Private and Public) 
4-Year Private 

4-Year Public 
Proprietary 

* Non-Semester 
Semester 

* Low Level of Automation 
High Level of Automation 

* Less Than 5 Professional 
Staff PTE's 

5 or More Professional 
Staff FTE*s 

* Less Than-4 Clerical or Data 
Entry Staff FTE's 

4 or More Clerical or Data 
Entry Staff FTE's 

Number of Pell Recipients 

* Little or No QC 
Mixed QC Procedures 
Automated QC Procedures 
Sampling QC Procedures 
Automated/Sampling QC 
Procedures 

Manual/Other QC Procedures 
Other/Sampling QC Procedures 



N/A 

Decrease 
Decrease 
Decrease 

N/A 

Decrease 
N/A 

Increase 



N/A 

Increase 
N/A 

Increase 
Decrease 
N/A 

Decrease 
Decrease 
Increase 

Decrease 
Decrease 
Decrease 



N/A 

1 percent 
1 percent 
1 percent 

N/A 

1 percent 
N/A 

1 percent 



N/A 

Not significant 
N/A 



1 percent 
1 percent 
N/A 

Not significant 
Not significant 
Not significant 

Not significant 
Not significant 
Not significant 



*N/A indicates that these characteristics were omitted from the 
regression and are captured in the intercept. 
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#«2^^2;-Bued Logistic ^agression Models 

As shown in Table 5-4, all QC categories are associated with lower 
rates of institutional Campus-Based overstatements of need relative to 
little or no QC procedures. In addition, all but one of these categories 
(manual/other QC procedures) is significant at the 5 percent level. As 
was the case with the Pell models, a high level of automation was 
associated with an increased rate of error relative to a low level of 
automation, yet the difference is not statistically significant. 

Table 5-5 presents the regression results for institutional 
campus-Based understatements of need. This table shows, again, that all 
QC categories are associated with a decrease in the rate of error 
relative to little or no QC procedures. However, only the categories 
sampling, other/sampling, and manual/other are significant at the 10 
percent level. In addition, the number of professional staff FTE's is 
significantly associated with institutional Campus-Based 

understatements. Five or more professional staff FTE's is associated 
with an increase in the probability of an institutional Campus-Based 
understatement relative to institution, with ewer than five professional 



staff FTE's. 



The regression results presented in Table 5-6 are those for 
institutional Campus-Based payment error. These results show that all QC 
categories are associated with lower rates of institutional Campus-Based 
payment error relative to the use of little or no QC procedures. The QC 
categories automated/sampling, other /sampling, mixed, and automated are 



TABLE 5-4 

LOGISTIC IBGRESSIOM ESTIMATES Of PBOBABILITT OF CAMPOS-BASED 
/mDc^k^ngniTS cw MIED. 1985-86 



SOURCE 



EFFECT CM 
PROBABILm 
OF EBBOR 



SI6MIFICAIICE 
LBVET. 



* 2-Yaar (Private and Public) 
4-Year Private 

4-Year Public 
Pr^^prietary 

* Non-Semester 
Semester 

* Low Level of Automation 
High Level of Automation 

* Less Than 5 Professional 
Staff FTE's 

5 or More Professional 
Staff FTE's 

* Less Than 4 Clerical or Data 
Entry Stiiff FTE's 

4 or More Clerical or Data 
Entry Staff PTE's 

Nuini')er of Campus-Based 
Recipients 

* Little or No QC 
Mixed QC Procedures 
Automated QC Procedures 
Sampling QC Procedures 
Automated/Sampling QC 
Procedures 

Manual /Other QC 
Procedures 
Other/Sampling QC 
Procedures 



N/A 

Decrease 
Increase 
Decrease 

N/A 

Decrease 
N/A 

Increase 



N/A 

Increase 
N/A 

Increase 

Decrease 
N/A 

Decrease 
Decrease 
Decrease 

Decrease 

Decrease 

Decrease 



N/A 

Not significant 
Not significant 
Not significant 

N/A 

Not significant 
N/A 

Not significant 



N/A 

Not significant 
N/A 

Not significant 

Not significant 
N/A 

i percent 
5 percent 
5 percent 

1 percent 

Not significant 

5 percent 



*N/A indicates that these characteristics were omitted from the 
regression and are captured in the intercept. 
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TMUJS 5-5 

LOGISTIC EBGWSSIOI BSTIIUTBS Of PIOBABILITT OF CAMPUS-BASED 
1MDB1STATIMBIITS OF MED, 19S&-86 



SOOTCg 

* 2-Year (Private and Public) 
4-Year Private 

4-Year Public 
Proprietary 

* Non-Semester 
Semester 

* Low Level of Automation 
High Level of Automation 

* Less Than 5 Professional 
Staff FTE*s 

5 or More Professional 
Staff PTE's 

* Less Thaji 4 Clerical or Data 
Entry Staff PTE's 

4 or More Clerical or Data 
Entry Staff PTE's 

Number of Campus-Based 
Recipients 

* Little or No QC 
Mixed QC Procedures 
Automated AC Procedures 
Sampling QC Procedures 
Automated/Sampling QC 
Procedures 
Manual/Other QC 
Procedures 
Other/Sampling QC 
Procedures 



BPFBCT m 
PEOBABILITT 
or EIBOB 

N/A 

Decrease 
Decrease 
Decrease 

N/A 

Decrease 
N/A 

Inc rease 



N/A 

Increase 
N/A 

Increase 

Decrease 
N/A 

Decrease 
Decrease 
Decrease 

Decrease 

Decrease 

Decrease 



SXGMIPICAMCB 
LIVIL 

HA 

1 percent 

Not significant 

Hot significant 

NA 

1 percent 
NA 

Hot significant 



NA 

5 percent 
NA 

Not significant 

1 percent 
NA 

Not significant 
Not significant 
5 percent 

Not significant 

10 percent 

1 percent 



*N/A indicates that these characteristics were omitted from the 
regression and are captured in the intercept. 



TABLE 5-6 



LOGISTIC IKaESSIM ISTIiaTSS Of PIOBABILm Of CAMPOS-BASED 



SDOBCE 



EFFECT m 
PSOBABILITT 
OF ElKOE 



SIGHIPICAIICE 
LEVEL 



2-Year (Private aac Pixblic) 
4 -Year Private 
4.Year Public 
Proprietary 

Non-Semester 
Semester 

Low Level of Automation 
High Level of Automation 

Less Than 5 Professional 
St^ff PTE's 

5 or More Professional 
Staff FTE*s 

Less Than 4 Clerical or D&ta 
Entry Staff PTE's 
4 or More Clerical or Data 
Entry Staff PTE's 

Number of Campus-Based 
Recipients 

Little or No QC 
Mixed QC «>rocedures 
Automated QC Procedures 
Sampling QC Procedures 
Automated/Sampling QC 
Procedures 

Manual/Other QC Procedures 
Other /Sampling QC 
Procedures 



N/A 

Increase 
Increase 
Increase 

N/A 

Increase 
N/A 

Decrease 



N/A 

Increase 
N/A 

Decrease 

Decrease 
N/A 

Decrease 
Decrease 
Decrease 

Decrease 
Decrease 

Decrease 



M 

H significant 
He- significant 
Not: significant 

NA 

Not significant 
MA 

Not significant 



NA 

Not significant 
NA 

Not significant 

Not significant 
NA 

5 percent 

10 percent 

Not significant 

1 percent 

Not significant 

5 percent 



*N/A indicates that these characteristics were omitted from the 
regression and are captured in the intercept. 
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all significant «t the 10 percent level. No other explanatory variables 
are significant, yet in this regression a high level of automation is 
associated with a lower rate of institutional Campus-Based payment error 
compared to a low level of automation. 

The institutional error rates in each respective error category of 
the institution profiles we developed are presented in Table 5-7 and 
summarized in Exhibit 5-1. No strict pattern exists across the profiles 
and there are wide variations within the error measures across the 
profiles. However, Profile 1 (high automation, manual or other/sampling 
QC) tends to have lower rates of error while Profiles 3 ( lo« automation, 
little or no QC), 4 (low automation, manual/other QC), and 6 (high 
automation, automated QC) tend to have higher rates of error. 

The regression results indicate the importance of institutional 
quality control procedures in controlling institutional error. In 
particular, procedures that include sampling seem to be effective. The 
clustering of procedural-type errors identified in Chapter 2 is a further 
indication that sampling should be an effective technique for controlling 
error. Since these errors tend to occur systematically at institutions, 
a relatively small sample could identify the problem. Taken together, 
this suggests that the Institutional Quality Control Pilot Project, which 
is based on sampling recipients, should continue and be further developed 
and refined. However, because the Pilot exists, at this point, for a 
limited number of institutions, ED may want to develop informational or 
technical assistance materials related to institutional quality control 
procedures for non-participating institutions. In addition, ED should 
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IMPUTED PfiOBABILITIBS OP ERSOR 
FOR SELECTED IIISTITUTIOH PROFILES^ 1985-86 



Institutional 
Error Measure 



Percent 



Profile* 



Percent Percent Percent Percent Percent 



h ill Overawards 4.2 

Pell Underawards 6.4 

Campus-Based Over- 
statements of Need 21.1 

Campus-Based Under- 
statements of Need 7.5 

Campus -Based Pay- 
ment Error (Awards 
in Excess of Need) 11.2 



15.7 33.7 45.9 

17.6 6.4 24.2 



15.4 



19.3 



6.0 



35.1 



34.2 



27.0 



29.4 



39.7 



14.8 



32.6 22.6 

13.0 29.9 

14.4 28.2 
34.0 31.5 

10.5 8.4 



* Profile 1 = "high automation, manual or other/sample QC" 
Profile 2 = "high automation, sampling/automated QC" 
Profile 3 = "low automation, little or no QC" 
PiTofile 4 = "low automation, manual/other QC" 
Profile 5 = "high automation, mixed QC" 
Profile 6 = "high automation, automated QC" 
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if A 



mm 




INSTfTUnON PROFILE ♦ 



P«ll Ov^mwards 
Ptil Und«(raward» 
OB Ov«rttat«m«nts 

Z **ndarstaUm§nts 
C-B Paymant Error 



^ Proflla 1 m ^liigh automation^ manual or cthar/aampia OC" 
v ProfHt 2 » litgh «utomatton» automatad/aampla QC*" 
'' proflla ,3 « ''low^automation, IKtia of no OC** 

Profile ^ 9 ''lOW automation^ manual/othar QC** 
\ Proflla 5 m ^high automation^ mlxad QC" 

Proflla 9 M "high automattoni automatad QC** 



EXHIBIT 5-1. ERROR PROBABILITIES ESTIMATED 
USING LOGISTIC REGRESSION 
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draft "Dear Colleague" letters profiling the importance of institutional 

quality control. Previous corrective actions concerning institutional 

quality cortrol include the drafting and dissemination of ED*s "QC 
Handbook" to institutions, 

5.2 IMSTITOTIOHAL VALIDATKW 

validation is the primary strategy employed to control student 
error. The Findings report presented evidence concerning the 
effectiveness of validation alonq three critical dimensions: 

♦ The ability to target validation to those applications 
containing errors 

♦ The ability of the validation process to eliminate errors on 
those applications selected for validation, and 

♦ Tb:* ability of the validation process to r>iduce the amount of 
e. '•or remaining in validated cas^r . 

In this section we go beyond Findings to identify factors that might 
influence the effectiveness of validation and would be good candidates 
:or possible corrective actions, 

5.2,1 Sujwtnary from Findings 

The following summarizes the major conclusions reached in Findings 
regarding validation: 
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Validation in the Pell Grant Program 

• Validation activities in the Pell program occurred for 80 
percent of the cases and was .successful in eliminating error 
from selected recipients, 

• Validation of those recipients selected by the Pell Processor 
was very successful in targeting and eliminating item 
discrepancies, although a residual level of discrepancies 
remained for all recipients, 

• About $85 million of error was removed prior to award from 
cases selected by the Pell Processor; however, the reduction 
in underawards was slightly greater than the reduction in 
overawards . 

• Institutions chose to validate two-thirds of the cases not 
selected by the Pell Processor, In cases they selected, 
institutions were successful in removing payment error but 
less so than in Pell selected cases. Institutions were not 
successful in identifying and eliminating item discrepancies. 

Validation in the Campus-Based Programs 

• Validation activities in the Campus-Based programs were about 
as extensive as in the Pell program and increased between 
Stage One and Stage Two in terms of both recipients selected 
by the Pel 1 Processor and those selected by institutions . 
Institutions used the Pell Processor flag to select 
recipients for verification and verified these recipients at 
a higher rate than non-flagged Campus-Based recipients. 

• Recipients receiving aid from multiple programs (e.g,. Pell 
and Campus-Based) were more likely to be verified by 
insti tut ions , 

• Campus-Based recipients selected for Pell validation have the 
lowest remaining error rates. 

• All validated cases have lower item discrepancy for adjusted 
gross income, although institutional verification is not 
successful at reducing item discrepancy in general. 

Validation in the GSL Program 

• Institutions validated Campus-Based and GSL recipients at a 
higher rate than recipients receiving only GSL's, 

• The percent of GSL rec ipients that are Pel 1 validated is 
extremely low. If GSL and Pell recipients are in error, they 
are likely to have a large error. 
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in general, the results indicate y:hat validation is an effective 
strategy for reducing student error in the Title IV programs. While this 
is an important finding, by itself it does not have any implications for 
corrective actions. If we can relate certain controllable factors to the 
effectiveness of validation, however, we may be able to identify methods 
for improving the ability of validation to control error. For example, 
if a given procedure is found to be related to more successful validation 
results, then the implementation of this procedure in situations where it 
is not currently in use might help improve quality. To accomplish this 
we need to explain the variation in the effectiveness of validation. 
This is similar to the problem of explaining the variation in 
insticut \onal error and, for the same reasons discussed in Section 5.1, 
also needs to be addressed using a multi"iriate model. 

5.2.2 Methodology and Analytic Approach 

The first step in the analysis was to define a measure of the 
effectiveness of validation. The two possible choices were error removed 
or error remaining. Error remaining, because it considers error only at 
the end of the award cycle, is not a ,ood measure of the effectiveness of 
validation. Error removed allows us to assess the extent to which 
accuracy was improved during the award process, the primary goal of 
validation. Consequently, the effectiveness of validation was defined in 
terms of the probability of an error that occurred at the beginning of 
the award process being removed by the end of the process. 
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The use of error removed as the measure of effectiveness meant that 
the analysis had to be limited to the Pell program. To determine if 
error was removed requires calculating error at both the beginning and 
the end of the award process. This requires access to a centralized 
applicant history file which is only available in the Pell program. The 
exact specification of the measure under consideration, the dependent 
variable, is a dichotomous variable coded one if a student error on the 
first CAR transaction was removed by the payment transaction and zero if 
the error on the first CAR transaction remained through to the payment 
transaction. Student error was defined as the difference between: the 
award calculated using data reported on the given transaction, best 
enrollment status, and best cost of attendance; and the best award. 

The factors hypothesized to be related to the probability of an error 
being removed in the Pell program, the explanatory variables, can be 
divided into three types: 

• In titutional characteristics 

Type and control 
Academic calendar 

o Student characteristics 

Effective family income 

Whether the application was filed before June 1. 1985 
Dependency status 
Tax filing status 

Dollar amount of award error at the beginning of the 
process 

« Validation-related procedures 
Validation status 

Professional full-time equivalent financial aid staff 
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C-lerical full-time equivalent financial aid staff 
fwg2>** to which verification tracking was automated 
— Whether professional staff conducted validation 
Average time spent in conducting validation 
Average number of contacts made in conducting validation 

The student and institutional characteristics included in the model 
were chosen from those found to be significantly related to error in the 
bivariate analysis presented in Chapter 6 of Findings. The validation- 
related procedures were activities associated with the validation process 
which could be altered by institutions and were hypothesized to be 
associated with the removal of error. 

The purpose of the analysis is to estimate the relationship between 
the validation-related procedures and error removed, controlling for the 
effects of student and institutional characteristics. In this way we can 
determine if there might be activities related to validation which ;;ould 
be altered by institutions to increase the ability of the system to 
remove error. The appropriate statistical technique for conducting this 
type of analysis, as stated previously, is multiple regression analysis. 
As was stated in section 5.1, because the dependent variable is discrete, 
logistic regression was used to estimate the equations. 



5.2.3 Results and Conclusions 



Table S-S presents che results of the logistic regression model. The 
results confirm that even in a multivariate analysis controlling for 
other factors, the probability of an error being removed is significantly 
higher for Pell selected cases than for not iielected cases. Institution 
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LOGISTIC tEGRBSSiai ESTIMATES Of PROBABILITy Of 
HAVIMG AM EtROR SENGVSD, i9aS-«e 



SOURCE 

Institutic ^ nal Characteristics 

* 2--Year (Private and Public) 
Proprietary 

4-Year Private 
4-year Public 

* Non-Semester 
Semester 

Student Characteristics 

Effective Family Income 

* Application Not filed 
by 6/1/85 

Application Filed by 
by 6/1/85' 

* Dependent 
Independent 

* Filed Single Tax Return 
Filed Joint Tax Return 
Did Not File Tax Return 

Error on First Transaction 

Validation Related Procedures 

* Not Validated 
Pell Validated 
Institution Validated 

* Less Than 5 Professional 
FTE's 

5 or More Professional 
FTE' s 

* Less Than 4 Clerical or Data 
Entry PTE's 

4 or More Clerical or Data 
Entry Entry FTE's 



EFFECT m 
PROTABILITT OF 
ERROR REMOVAL 



N/A 

Decrease 
Decrease 
Increase 

ecrease 
Decrease 



Decrease 
N/A 

Decrease 
N/A 

Increase 
N/A 

Decrease 
Decrease 

Increase 



N/A 

Increase 
Increase 



N/A 

Increase 
N/A 

Increase 



*N/A indicates that these characteristics were omitted 
regression and are captured in the intercept. 



SXGHIFICAMCE 
LEVEL 



N/A 

Not significant 
Not significant 
Not significant 

Not significant 
Hot significant 



Not significant 
N/A 

1 percent 
N/A 

1 percent 
N/A 

Hot significant 
Not significant 

1 percent 



N/A 

1 percent 

Not significant 



N/A 

Not significant 
N/A 

Not significant 
from the 



TABLB 5-8 (CoatiaiMd) 
LOGISTIC BBGKESSIOH ISTIIUTBS OF PKOBXBILITY OF 



SOURCE 



EFFECT CM 
PSCXBABILITT OF 
EBRORSmOVAL 



Validation Related Procedures (Cont>)_ 

* Validation Tracking 

Not Automated N/A 
Validation Tracking 

Partially Automated Decrease 
Validation Tracking 

Fully Automated Decrease 

* Non-Professional Staff 

Conduct Validation N/A 
Professional Staff 

Conduct Validation Decrease 

* Less Than 10 Minutes 

Per Validation N/A 

14-20 Minutes Per Validation Increase 

21-30 Mixfutes Per Validation Increase 

Over 30 Minutes Per Validation Increase 

* Vour or More Con^acts Per 
Validation N/A 
Three Contacts Per Validation Increase 
Two or less Contacts Per 

Validation Increase 



SIOIIFXCAMCE 
LEVEL 



N/A 

Not significant 
Not significant 

N/A 

Not significant 



N/A 

Not significant 
Not significant 
Not significant 



N/A 

Not significant 
Not significant 



*N/A indicates that these characteristics were omitted from the 
regression and are captured in the intercept. 
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selected cssss also have a higher probability of having an error removed 
than not selected cases, although the difference is not statistically 
significant* 

Besides confirming the bivariate analysis presented in Findings that 
validation, particularly of cases selected by the Central Processor, was 
successful at removing error, there are few significant results. None of 
the institutional procedures which were felt might improve the 
effectiveness of validation are found to be significantly related to 
higher probabilities of having an error removed. Higher levels of 
professional and clerical staff size, automation of verification 
tracking, having professional staff conduct validation, and increased 
time and number of contacts employed in conducting validation are not 
related to higher probabilities of error removal, contrary to what was 
hypothesized. Institutional characteristics, such as type anJ control of 
institution and whether a semester system was used, are also not related 
to the probability of having an error removed. 

In addition to validation, error removal is only found '^o be related 
to certain student characteristics- Independent students are more likely 
to have an error removed, while students filing their aid applications 
before June 1, 1985 are less likely to have had an error removed. The 
higher the air^ount of error at the beginning of the process, the higher 
the probability that the error was removed. This suggests that 
institution? were more conscientious about conducting validation, if 
there appeared to be large errors present. 



The (model) results lead to two conclusions. First, there is an 
increased probability of having an error removed if a recipient were 
selected for validation by the Central Processor, as opposed to 
optionally by the institution. This indicates that the implementation of 
the integrated verification criteria for Pell non-recipients should 
increase the amount of error removed, even if the percent of Pell 
non-recipients verified remains the same. Just switching from voluntary 
verification of Pell non-recipients to the integrated verification 
criteria should improve error removal. The superior results obtained 
given selection by the Central Processor for Pell recipients should also 
be obtained for non-Pell recipients selected through the integrated 
verification criteria, given the similarities between the two processes. 

The second conclusion concerns the fact that there appears to be no 
relatioa-:hi? between controllable institutional procedures related to 
validation and the probability of an error having been removed. This 
suggests that there are no readily apparent corrective actions for 
improving the effectiveness of validation without changing the nature of 
the validation process itself. If certain institutional procedures had 
been related to higher probabilities of error removal, then wider 
implementation of these procedures might have improved the effectiveness 
of validation without requiring alteration to the process. Procedures 
that, had they been significantly related to removing an error, could 
have been implemented include: 

« The type of validation tracking an institution used 

• The type of staff conducting validation, or 

• The time or number of contacts spent validating cases. 
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However, because no procedure of this type is identifiable from the 



validation necessitates improving the process for selecting students for 
validation- 

In Chapter 2, we showed how the use of tax forms could also help 
identify and possibly limit errors in home value, savings, and 
dependent's net assets. Since these data items are not currently part of 
the validation process, adding them is a potential method for improving 
the effectiveness of validation. However, this would also increase the 
burden placed on institutions. Given that 72 percent of institutions, in 
response to a question posed in the Institutional Questionnaire, felt 
that the average time spent on validation had increased in 1985-86, there 
would likely be much dissatisfaction with any proposal further expanding 
the requirements of validation. Again, better targeting of error-prone 
cases is a cost-effective solution. 



results or the model, further improvement 



in the effectiveness of 
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SIMPLIFICATION OP ME UNIFOBM METBODOLOGY 



This chapter presents the results of our analyses of the effects of 
reducing the number of data elements used in the Uniform Methodology (UM) 
formula. The UM formula is used to determine a student's Expected Family 
Contribution (EFC) in the Campus-Bassd programs. Seducing the data 
elements that comprise the UM is a Level IV corrective action. 
Significantly restructuring the UM formula will affect the largest 
portion of residual error of all corrective actions presented in this 
report. In general, our analyses indicated the following concerning 
truncating the UM formula: 

• The distributions of need in the Campus-Based programs and 
certification in the GSL programs resulting from the reduced 
formula closely approximates the intended distributions 

e Most recipients have minimal changes in their Campus-Based 
need and GSL certification under the reduced formula 

• Student error in the Campus-Based and GSL programs is 
significantly lower under the reduced formula, and 

• Adjusting aspects of the truncated formula could improve on 
the effectiveness of the reduced formula. 

These findings are discussed in detail in the following sections. In 

addition a paper entitled "Need Analysis: Thoughts For Reform," authored 

by the NASFAA Need Analysis Standards Committee, is reproduced in 
Appendix C. 
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S.l £ U*g4> _gy FBQM PBTQIt AMALYSIS 

The findings from numerous quality control studi«*s, and a pervasive 
perception that student aid in general, and the formulae used to 
determine eligibility and need for Federal financial aid funds in 
particular, are too complex, has led ,o a widespread interest in 
simplifying tl-3se formulae. In Stage One of the current study, ED 
conducted a multifaceted assessment of Pell simplification which focused 
on equity and quality issues. This assessment employed an analytic 
framework to evaluate and rank individual data elements across key 
criteria (e.g., budgetary, distributional, reliability etc.). The 
ranking produced a set of data elements that could be eliminated from the 
formula with little impact across numerous dimensions. 

In keeping with the dual focus on equity and quality, the Stage One 
analysis of simplifying the Pell Family Contribution Schedule (FCS) was 
the first that took into account both adjusting the formula and 
controlling for reporting error. This analysis of a six element formula 
led policymakers to conclude that prior analyses, which did not control 
for reporting errors, overstated the distributional and budgetary effects 
of simplification. In the analysis that used verified values, the 
negative budgetary and distributional consequences of simplification were 
reduced substantially.. 

These findings provided the impetus for Congressional action in the 
recent reauthorization of the Higher Educa'.ion Act. The reauthorized Act 
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now requires the Secretary of Education to produce a six element Pell 
form for families with income under $15,000. 

In addition to the recent Congressional action, the Stage Two 
findings indicating high levels of reporting error in the Campus -Based 
programs have kindled interest in the potential for simplifying the 
Uniform Methodology (UM), the system for determining the expected family 
contribution (EFC) for the Campus-Based programs. 

The potential advantages of, and arguments for, considering 
simplification of the UM are manifold. First, simplification can reduce 
the length and complexity of forms. This change would enhance applicant 
understanding and perhaps reduce inadvertent mis reporting. It would also 
reduce applicant burden and certain processing costs. In addition, a 
shortened formula couid increase understanding on the part of applicants 
and some student aid personnel of how programs distribute aid and also 
reduce institutional burden associated with verification and 
institutional processing costs in general. From a system-wide 
perspective, a shortened UM formula ma^es integration of the Title IV 
programs more feasible and easily accomplished. 

In many ways the arguments for simplification are stronger with the 
UM than with the FCS . The loose tie between need and awards suggests 
that small to rjderate changes in the distribution of need could occur as 
a result of simplification with little impact on Campus-BasM awards. In 
Pell, however, the relationship between SAI and award is direct, and. 
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I thus, changes in the distribution of SAI ' s that result from 

? simplification have a greater impact on Pell awards. 

Perhaps the single disadvantage of UM simplification is the potential 
* for significant distributional consequences. If these changes cause 

large need shifts, or shifts in different directions for different 
subpopulations . Campus -Based awards could be affected for these 
; subpopulat ions . 

In this chapter we explore whether reporting error in the 
Campus-Based programs is serious enough to consider simplifying the UM. 
Although the answer no this question may appear self-evident to some in 
light of the Findings volume, the measures used to describe the errors 
are high level aggre'^ations and tend to obscure the distributional 
effects of error on different subpopulations . Thus, we will analyze the 
effects of error by examining how it changes need across various 
subpopulations. We do so first by addressing the question: How far from 
an equity benchmark is the current system, and what is the direction of 
the deviation? An equity benchmark in this analysis can be created by 
developing a distribution of need with the full UM formula and verified 
data. Substantia'^ deviation of the current system from this benchmark 
would argue strongly for simplifying the UM, especially if the target 
population (i.e., low income recipients) were being hurt by the current 
system. 

In that case a second set of questions is appropriate. First, what 
: would be the effect of truncating the UM and using a six element formula 



ciniilar to the simplified Pell formula. Second, would an alternative 
formula be required. These questions are explored in the following 



sections . 



Simplification, by its nature-, is a long-term corrective action. 
Because of impending policy changes requirad by reauthorization, 
simplification will be implemented under conditions that are different 
from current procedures. The change from prospe tive to base year income 
constitutes a major change that must be controlled for, if the present 
analysis is to be a useful guide to policymalcers . Thus, this simulation 
analyzes the effects of simplication under the conditions in which it 
would be implemented. 

6.2 METHODOLOGY AND ANALYTIC APPROACH 

The following sections describe the procedures we used to analyze 
simplification of the UM formula. We first describe how we defined a 
shortened formula, and then discuss how we used alternative models to 
simulate the effects of this shortenea formula and the criteria used to 
evaluate the shortened formula- 

6.2.1 Forwula Redefinition 

The UM methodology was redefined to include only those items included 
in the reduced Pell formula defined in -Data Element Reduction." The 
remaining data items are: 
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• Adjusted gross income 

• Federal taxes 

• Other non-taxable income 

• Household size, and 

• Number in college. 

Despite the use of identical data elements, simplification will have 
substantially different effects on the Campus-Based population, due 
largely to the differences between the Pell Family Contribution Schedule 
and the UM. Income items in the reduced EFC formula reflect base year 
values for both dependents and independents. Base year income values 
were used to capture the change in regulations requiring the use of base 
year income rather than prospective income for independent studants under 
the Uniform Methodology. The change in regulations also specified that 
an EFC be calculated, using the Uniform Methodology, for all GSL 
recipients regardless of Family AGI . This also means that Campus-Based 
recipient., with Family AGI's of S30,000 or less can no longer borrow 
their EFC. 

The following summarizes the effects of eliminating the selected data 
items from the formula: 

o Income portions substituted for AGI for non-filers 

• Social security benefits and AFDC included in Other 
Non-Taxable Income 

• Assumptions based on eliminated data items no longer valid 

• Summer Savings set to $900 for dependents, $0 for 
;ndependents 

• Dv>pendent applicants rejected if no income or household size 
reported 

• Educational VA benefits excluded from formula 

• Federal Tax computation eliminated from formula 

• FICA tax figured on total AGI and capped at $3,800 



State tax rate assumed 4 percent for dependent parents and 0 
for independents 

Medical, Tuition, and Employment Expense Offsets eliminated 
Maximum contribution from total income based solely on 
household size 

No student contribution for dependents, and 
Assets eliminated from formula. 



The two most pronounced effects of the reduction in the number of 
data elements is the elimination of several expense offsets and the 
contribution from assets. The elimination of offsets to income increases 
EFC and, thereby, decreases need. In the aggregate, for independent 
students, the removal of assets from the formula tends to counter the 
effect of eliminating income offsets. Recipients whose net assets exceed 
the protection allowance (which is based on age) under the current 
formula show a decrease in EFC and increase in need under data element 
reduction. Since the vast majority of independent students had 
contributions from assets of less than $100, the impact of eliminating 
assets is limited to a relatively small number of students. 

The impact of eliminating assets on dependent students is not as 
straightforward, however. Under the current formula, assets can reduce 
EFC for certain dependent students. The parents of dependent studen«-s 
whose net worth is negative (i.e., net assets are less than the 
protection allowance) can have a negative contribution from assets. If 
net worth is negative and available income (income minus offsets) is less 
than $15,000, then available resources and, hence, parental contribution 
and EfC are reduced lather than increased by the contribution from 
assets. Removing assets from the formula for dependent students can, 
therefore, increase EFC and reduce need for some students, -hile 
decreasing EFC and increasing need for others. 
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Table 6-1 indicates that for nearly 60 percent of dependent students, 
assets act to reduce EFC under the current formula. The negative 
contribution from assets is most prevalent among the lowest income group 
and diminishes as income rises. Consequently, for a large number of 
dependent students, particularly low income students, the elimination of 
assets from the formula causes EFC to rise and need to fall. Dependent 
students- own assets can only increase EFC under the current formula. 
However, as with independent students, nearly all dependent students had 
contributions from their own assets of less than $100. Thus, elimination 
of dependent studenr assets is likely to have a small impact on need. 

m conclusion, assets often do not result in increases in EFC and, 
for dependent students, may result in decreases under the current 
formula. Hence, eliminating assets, based on the analysis, does not 
cause large decreases in EFC and increases in need. This, in conjunction 
with the dscrea.ses in need caused by the elimination of several income 
offsets, means that need should most likely decrease, using reported 
data, in the aggregate due to data element reduction. 

6.2.2 Model Definition and Evaluative Criteria 

Combining the two formulae (full and reduced) with the two levels of 
data (best and reported) yields four models for analysis and comparison: 

• Full formula using best data 

• Full formula using reported data 

• Reduced formula using best data, and 

• Reduced formula using reported data. 
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TABLE 6-1 



IMPACT OF ASSETS OH DEPENDENT STUDENTS' EFC UNDER 
THE CURRENT FORMULA, 1985-86 



INCOME 
LEVEL 



$10,000 or less 

$10,001 - ^15,000 

$15,C01 - $25,000 

* Greater than $25,000 

** All Recipients 



ASSETS 
DECREASE 
EFC 

PERCENT MEAN 

75.0 1,565 

71.3 1,427 

66.4 1,071 
40.9 450 

59.7 1,102 



ASSETS 
INCREASE 
ETC 

PERCENT MEAN 

25.0 3,501 

28.7 2,754 

33.6 3,240 

49.6 4,035 

36.9 3,599 



* In 9.5% of cases there was no effect on EFC 
In 3.4% of cases there was no effect on EFC 



As defined. EFC determined under data element reduction can differ 
from that obtained under the current formula for three reasons: 

• The elimination of data elements 

• The use of base year rather than prospective income for 
independent students, or 

• The calculation of an EFC for all GSL recipients regardless 
of family AGI. 

The latter two differences result from changes already specified in the 
regulations and are not directly related to data element reduction. In 
addition, the change from prospective to base year income was already 
analyzed in Chapter 3. Measuring just the impact of data element 
reduction, therefore, requires eliminating the effects of these other two 
changes . 

TO focus our analysis specifically on the impact of data element 
reduction, we redefined the full best and full reported models to include 
the changes specified by the new regulations - base year income used for 
all students and EFC's calculated, using the UM. for all GSL recipients. 
In this way. the difference between the full and reduced models is only 
caused by the elimination of data elements. Redefining the full models 
means that, unlike in Chapters 3 and 4. the full reported model is no 
longer observed, but rather is simulated. Consequently, students can be 
found to have zero need under the full reported model even though they 
actually received Campus-Based aid and/or a GSL. (This could also happen 
due to the problem of initial overawards discussed in Chapter 5 of 
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Findings >) Also, the error rate occurring under the full model (i.e., 
full best compared to full reported) is based on the changed regulations 
{e.g., income error is base year for independents rather than prospective 
income), and hence does not equal the error rates found for the current 
formula presented in Findings . 

Campus-Based need and GSL certification are computed under each 
model. Pell awards are also computed for each of the four models because 
Pell eligibility changes under data element reduction and this change 
also affects need for the Campus-Based and GSL programs. Campus-Based 
need for each model is determined by subtracting the EFC, the Pell award, 
and, if packaged before Campus-Based, the GSL award computed for that 
model, and other aid from the cost of attendance. The GSL certification 
for each model is determined by subtracting the EFC, Pell, and 
Campus-Based awards, if packaged before GSL, computed under that model 
and other aid from the cost of attendance. The Campus-Based award for 
GSL purposes is taken as the minimum of the actual Campus-Based award and 
the Campus-Based need coir ed under the given model. Other aid and cost 
of attendance are held constant across all models at the values 
originally used by the institution ("reported" values). 

As many as six comparisons of an- two models can be made among the 
four. The subsequent sections of this chapter discuss the more 
meaningful of these comparisons. The three comparisons which include the 
full best model indicate the relative merit of each of the other three 
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models based on similarity to optimality. Comparisons made by holding 
the level of data constant measure the impact of the reduction of data 
elements. Holding data at reported values captures the Impact of data 
element reduction under the current level of data. Holding data at best 
values captures the -true" impact of data element reduction. Comparisons 
made by holding the formula constant and varying the level of data 
measure the effect of student misreporting on each of the two formulae. 
These effects can then be compared to assess the validity of the two 
formulae. Model closeness, joint distributions of need or certification, 
summaries of student error, and analysis of loss and gain of need and 
certification are discussed in the sections which follow. 

6.3 RESULTS "AND CONCLUSIONS: CAMPUS-BASED MODELS 

The following sections present an assessment of the relative 
performance of the Campus-Based models according to several different 
measures. These measures include how effective the alternatives are in 
distributing need among the intended recipients, what the impact on need 
is under the alternative models, and what student need error would be 
under the reduced formula compared to the full formula. All of these 
measures should be assessed when eva -lating the reduced formula. 

6.3.1 Heed Distribution Under the Caapus-Based Models 

The degree to which a given UM model assesses need in the 
Campus-Based programs in a manner similar to the intended or optimal 
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assessment, can be determined by looking at the percent of total need 
represented by various income groups under the different models. The 
intended assessment is the distribution of need obtained under the full 
best model. Table 6-2 presents these figures for dependent recipients 
and Table 6-3 shows the analogous figures for independent students. 

As Table 6-2 shows, the intended distribution of Campus-Based need 
for dependent students is that families with incomes of SIO.OOO or less 
should represent 21.6 percent of total need, famiHes with incomes of 
$10,001 to $15,000 have 17.8 percent, those with $15,001 to $25,000 
income have 38.7 percent, and families with incomes of more than $25,000 
should represent 21.8 percent of total need. The differences between 
these percentages and the percentages for the full formula using reported 
data show the extent to which reporting error redistributes total need 
away from the intended distribution. These figures show that reporting 
error redistributes need from the lowest three income categories to those 
families with incomes greater than $25,000. This can be seen by the fact 
that families with $10,000 or less in income drop from 21.6 percent of 
need under the full best model to 19.0 percent under full reported. 
Families with $10,001 to $15,000 go from 17.8 percent to 16.1, those with 
$15,001 to $25,000 from 38.7 percent to 37.7, and the highest group from 
21.8 percent to 27.2 percen*" 

Using a reduced formula with best data moves the distribution closer 
to the intended distribution than the full reported model. The 
percentages of need thst the income groups represent under the reduced 
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TABLE 6-2 



PERCENT OF TOTAL HEED FOR CAMT'TS-BASED DEPEMDEHT RECIPIENTS 
BY INCOME LEVEL UNDER ALTERNATIVE MODELS, 1985-86 



INCOME 
LEVEL 




FUI 
BESi 


FULL 
REPORTED 


REDUCED 
BEST 


REDUCED 
REPORTED 


$10,000 


or less 


21.6 


19.0 


20. 8 


19.1 


$10,001 


. $15,000 


17.8 


16. 1 


16.7 


16.6 


$15,001 


- $25,000 


38.7 


37.7 


36. 9 


35.9 


Greater 


than $25,000 


21.8 


27.2 


25.6 


28.5 



TABLE 6-3 



PERCENT OF TOTAL NEED FOR CAMPUS-BASED INDEPENDENT RECIPIENTS 
BY INCOME LEVEL UNDER ALTERNATIVE MODELS, 1985-86 



INCOME 
LEVEL 


FULL 
BEST 


FULL 
REPORTED 


REDUCED 
BEST 


REDUCED 
REPORTED 


$2,000 or less 


29.8 


27.0 


37.1 


35.3 


$2,001 - $4,000 


18.2 


17.0 


16.6 


16.9 


$4,001 or $8,000 


33.9 


35.2 


27 .7 


27.6 


Greater than $8,000 


le.i 


20.9 


18.7 


20.3 
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best model, as compared to full reported model, shift need from the 
highest two income groups to the two lowest. While the amount of this 
Shift is not enough to compensate for the reporting error under the full 
formula, the distribution under the reduced best model is closer to the 
intended distribution than the distribution under the full reported model 
in three of the four income level groups. 

A similar pattern also exists among the four models for independent 
students. However, under the reduced best model, the lowest income 
students gain in percentage of need while students in the second and 
third groups have slightly smaller percentages of need. 

Table 6-4 shows a composite analysis of dependent and independent 
students of the percentages of need each income group represents. The 
income groups have been held constant in this table for both independent 
and dependent recipients. Thus, the lowest income group in Table 6-4 
includes dependent students with family income of $10,000 or less and 
independent recipients with income of $2,000 or less. This composite 
table shows that the reduced best formula comes very close to the 
intended distribution, but that the lowest and highest income groups gain 
Slightly (from 24.5 percent to 25.3, and from 20.5 to 23.7 respectively)^ 
in percentage of need, while the second and third income groups lose> 
slightly (fron 18.0 to 16.6 and from 37.0 to 34.4). Therefore, reducing 
the UM formula shows promise in achieving a distribution of need closer 
to the intended distribution than the current distribution. 
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TABLE 6-4 



PERCENT OF TOTAL NEED FOR ALL CAHPUS-EASED BECIPIBHTS 
BY INCOME GROUPS UNDER ALTERNATIVE MODELS, 1985-86 



INCOME 
GROUP 

Lowest 
Second 
Third 
Highest 



FULL 


FULL 


REDUCED 


BEST 


REPORTED 


BEST 


24.5 


21.8 


25.3 


18.0 


16.4 


16.6 


37.0 


36.8 


34.4 


20.5 


25.0 


23.7 



REDUCED 
REPORTED 

23.7 
16.7 
33.5 
26.1 
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The amount of total need changes under each of the four models. 
Table 6-5 shows that true need is $3,096.2 million and reporting error 
overstates need by nearly $400 million. Need under the reduced formula 
comes very close to true need, and reporting error under the reduced 
formula (approximately $40 million) causes less of a shift away from true 
need (than reporting error under the full formula). 

6.3.2 Need Awounts Under the Ca«pus -Based Models 

In addition to measuring the degree to which total need is 
distributed under the alternative models, one should consider the 
direction of the flow of need among recipients. The block charts 
presented as "Exhibits 6-1 and 6-2 illustrate the change in need occurring 
when comparing a pair of models. The axes define groups of recipients by 
need values under the respective models. The height of the block in each 
of the cells indicates the percentage of recipients in the cell. Blocks 
lying along the diagonal running from lower left to upper right represent 
recipients with small or no change in need. Blocks lying directly off of 
the diagonal represent recipients with moderate need changes, while 
blocks in the upper left and lower right corners represent recipients 
with extreme need changes. Blocks above the diagonal indicate higher 
need for the model represented by the vertical axis. Conversely, blocks 
below the diagonal indicate higher need for the modv -epresented by the 
horizontal ajsis. 



The block charts presented i^.-strate two comparisons of changes 
under the full and reduced models. These two comparisons are the 



TABLE 6-5 



TOTAL CAMPUS-BASED NEED UHDER ALTERNATIVE MODELS, 1985-86 

j^DEL TOTAL NEED (S Millions) 

Full 3est 3,096-2 

Full Keported 3,493.4 

Reduced Best 3, 132.7 

Reduced Reported 3,173.5 
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EXHIBIT 6-1. CAMPUS-BASED NEED UNDER FULL AND REDUCED UM MODELS USING REPORTED DATA 

(PERCENTAGE BLOCK CHART) 
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EXHIBIT 6-2. COMPARISON OF CAMPUS-BASED NEED UNDER FULL AND REDUCED UM MODELS USING B?.3T DATA 

(PERCENTAGE BLOCK CHART) 
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apparent and true effects of shortening the formula. The apparent 

effects can be determined by looking at how need changes using reported 

data and the true effects can be assessed by looking at the full and 
reduced models using best data. 

Exhibit 6-1 shows the distribution of Campus-Based need under the 
full reported model along the vertical azis, paired with reduced reported 
model, along the horizontal ajtis. This pair represents the apparent 
impact of data element reduction on Campus-Based need. The chart shows 
taller blocks below the diagonal than below, with 66 percent of 
recipients receiving roughly the same award under both formulae. Exhibit 
6-1 also shows that roughly 21 percent of recipients' need decreases 
under <-.he rftduced formula while only 13 percent of recipients increase 
need under the reduced model. 

Exhibit 6-2 displays the distribution of Campus-Based need comparing 
full best model along the horizontal axis with reduced best model along 
the vertical axis. This represents the true impact of data element 
reduction on Campus-Based need. As in Exhibit 6-1, the blocks are 
tallest along the diagonal. Exhibit 6-2 shows that approximately 20 
percent of recipients gain in need and 15 percent have their need 
reduced. Disputing the apparent effect of reducing the number of data 
elements the (difference between full reported and reduced reported). 
Exhibit 6-2 shows that the true effect (the difference between full best 
and reduced best) is a modest increase in need rather than a decrease. 



6.3.3 Student Heed Error Under the Campus-Based Models 



In analyzing the effects of data element reduction it is helpful to 
compare the effects of student misreporting on need under the full and 
reduced formulae. Table 6-6 summarizes Campus-Based student need error 
under the full and reduced models. The overall error rate is nearly 55 
percent under the full formula. (This result is not comparable to 
analyses in Findings because base year income was substituted for 
prospective income.) The reduced formula has an overall error rate of 
just over 41 percent. The average understatement is $159 higher under 
the reduced formula. Total understatements are $23.4 million higher 
under the reduced formula and the overstatement rate is 13.1 percentage 
points higher ander the full formula. The average overstatement is 
nearly $400 higher under the full formula, and overstatements total 
$328.2 million less under the reduced formula. Total net need error is 
almost 10 times higher under the full formula. 

6.3.4 Conclusions of Campus-Based Alternatives 

By looking at the evaluative criteria just discussed, shortening the 
UM formula shows promise in increasing the accuracy ano efficiency of 
determining need in the Campus-Based programs. The distribution cf need 
under a reduced formula overcomes to a degree the distortions from the 
intended distribution caused by student reporting error. The 
distribution of need under the reduced model appears especially 
attractive since the target group of recipients, those in the lowest 
income groups for independent and dependent recipients, gain slightly 
relative to tne other income groups. 
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CANPUS-BASCO SlUOEin NEEO EMNM UNDER THE FULL ADO REDUCED FOmJLAE\ I96S-86 



UMDEBSTAIEMEWTS 



(WitMfi $S0) 



foial 



OVERSTAIEHEMyS 



foUl 

Eeixfiot tkio iSJfiiliflal 



MPT MTFII 



Full Formula 45.5 
Reduced Formula 58.8 



15.7 849 
15.4 1.008 



170.7 
199.1 



38.8 1.143 567.4 
25.7 726 239.2 



310 
31 



396.7 
40.1 



I 

U) 



* Deleting prospective income for all recipients. 
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Furthermore, because the amount of need recipients have under the two 
models is nearly the same over 60 percent of time, and the amount of 
student need error under the reduced formula decreases dramatically, the 
possibility of the reduced formula achieving its goals is significant. 
The ultimate reduced formula might not take the sa-ne form or have the 
same data elements as the reduced formula used for these analyses. 
However, because simply truncating the formula produced such positive 
results, slight modifications to the reduced formula (either in its data 
elements or the tax rates used) could increase the effectiveness of a 
reduced element formula even more. 

6.4 RESULTS AMD CONCLUSICWS: GSL MODELS 

The following sections present the results of analyzing the effects 
of reducing the Ul-i f=r-ul2 in the nsr. program. The evaluative a.-v£lyses 
we present are the same as those presented for the Campus -Based 
programs. We will first show the effects on the distribution of 
certification by income level, then the impact on certification amount, 
and finally student error under the full and reduced formulae. 

6.4.1 Certification Distribution Under the GSL Models 

Tables 6-7 and 6-8 show the distributional effects on GSL 
certifications of shortening the UM. Table 6-7 presents the figures for 
dependents and Table 6-8 for independents. (Table 6-9 collapses across 
dependency status.) Table 6-7 shows that for dependent recipients, the 
reduced model with best data comes closer to the benchmark than using a 
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PERCENT OP TOTAL CBRTIFICATIOH FOR GSL DEPENDENT RECIPIENTS 
BY INCOME LEVEL UNDER ALTERNATIVE MODELS, 1985-86 



INCOME FULL 

LEVEL BEST 

$10,000 or less 20.0 

$10,001 - $15,000 13.7 

$15,001 - $25,000 24.1 

Greater than $25,000 42.2 



FULL 
REPORTED 

16.2 
11.7 
22.3 
49.8 



REDUCED 
BEST 

20.0 
12.0 
24.3 
43.6 



REDUCED 
REPORTED 

18.4 
12.0 
24.3 
45.3 



TABLE 6-8 



PERCENT OF TOTAL CERTIFICATION FOR GSL INDEPENDENT RECIPIENTS 
BY INCOME LEVEL UNDER ALTERNATIVE MODELS, 1985-86 



INCOME 
LEVEL 



$2,000 or less 
$2,001 - $4,000 
$4,001 or $3,000 
Greater than $8,000 



FULL FULL 

BEST REPORTED 

27.6 21.9 

15.9 12.7 

2;. 4 22 .0 

32.2 43.4 



REDUCED REDUCED 
BEST REPORTED 



35. L 27.9 

16.1 U.3 

19.4 18.0 

29.4 39.8 



TABLE 6-9 



PERCENT OF TOTAL CERTIFICATION FOR ALL GSL RECIPIENTS 
BY INCOME GRCXJPS UNDER ALTERNATIVE MODELS, 1985-86 



INCOME 
GROUP 

Lowest 
Second 
Third 
Highest 



14 
24 
38 



FULL 
BEST 

2.9 

,6 
,2 
.3 



FULL 
REPORTED 



18.5 
12.1 
22.3 
47. 2 



REDUCED 
BEST 



REDUCED 
REPORTED 



24.9 
13.4 
22.7 
39.0 



21.8 
12.8 
22.1 
43.3 



full formula with reported data. While under the reduced best model, 
dependents with family incomes of greater than S25,000 gain slightly 
(from 42.2 percent of certifications to 43.6 percent), the eKtent of the 
gain is less than occurs due to reporting error under the full formula (a 
7.6 percentage point difference). Dependent students with family incomes 
of $10,000 or less receive 20 percent of total certifications under both 
full and reduced formulae. 

The results for independents shown in Table 6-8, indicate a 
redistribution from the two highest income groups to the two lowest under 
the reduced best model. The lowest income group of independents 
increases from 27.6 percent of total certification to 35.1 percent under 
the reduced best model. Reporting error under the full formula causes a 
large amount of redistribution in the opposite direction; from the lowest 
three income levels to the highest income group which increases from 32.2 
percent to 43.4 percent. 

Table 6-9 presents the effects when collapsing by income group. In 
general, the data in this table suggest that, in the aggregate, a reduced 
element formula closely approximates the benchmark model. While the 
highest and lowest income groups gain slightly, the distribution under 
the reduced formula is closer to the intended distribution than the full 
reported model in each of the four categories. The full reported model 
redistributes certification from the three lowest income groups to the 
highest group. 
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Just as total need changed under the Campus -Based models, total 
certification will also change under the GSL models. Table 6-iO shows 
that reporting error causes total certification to differ dramatically 
under the full formula. This table also shows that certification amounts 
under the reduced models are significantly closer to the intended 
certification than full formula reported certification. In addition, the 
amount of reporting error is much less under the reduced model. The 
amount of certification presented in Table 6-10 does not agree with the 
figures presented in Findings due to the changes simulated based on 
reauthorization. The most critical of the changes is that the figures in 
Table 6-10 assume that all recipients will go through need analysis. 

TABLE 6-10 

TOTAL GSL CERTIFICATION UNDER ALTERNATIVE MODELS*, 1985-86 

MODEL TOTAL CERTIFICATION** 

($ Millions) 



Full Best 4,221.7 

Full Reported 5,314.5 

Reduced Best 4,442.2 

Reduced Reported a, 650.2 



* All recipients go through need analysis. 

** Due to a revision in the estimate of total GSL loan volume, these figures 
should be reductJ by approximately 10 percent. 
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6.4.2 Certification Amounts Under the GSL Models 

Exhibit 6-3 illustrates the apparent effects of shortening the 
formulas for undergraduates in the GSL program by showing the 
distribution of GSL certifications under the full reported model along 
the horizontal axis and the reduced reported model along the vertical 
axis. As with the Campus-Based models, the apparent effects of 
shortening the formula is to reduce certifications. Exhibit 6-3 shows 
that under a reduced formula roughly 23 percent of GSL certifications 
would decrease and 13 percent would increase (65 percent would have 
little or no change). 

Exhibit 6-4 represents the true af ects of shortening the formula for 
undergraduates by showing the distribution of GGL certifications under 
the full best model along the horizontal axis and the full reported model 
along the vertical axis. Exhibit 6-4 shows that the true effects of 
shortening the formula for undergraduate GSL certifications is that 10 
percent decrease, 17 percent increase, and 73 percent have little or no 
change . 

Exhibits 6-5 and 6-6 show the same distributions represented by 
Exhibits 6-3 and 6-4 respectively, but include only graduate students. 
In Exhibit 6-5, which shows the apparent effects, nearly 17 percent of 
graduate students nave decreased certifications while only 8 percent have 
increased certifications. Exhibit 6-6, representing the true effects, 
shows that 15 percent of graduate students have decreased certifications 
and 14 percent have increased certifications. Of those graduate students 



6-28 



V 




m on Le8» loi « im mi « 

QSL CEimriCATION U^Dkft TH6 
rUU» mPORTED, MODEL (t) 




EXHrerr 6-3. comparison of GSL CERIinCATtON UNDER 
FULL AND REDUCED UM MODELS USING REPORTED DATA 
FOR UNDERGRADUATE STUDENTS 
^ o r (PERCENTAGE BLOCK CHART) 



I 

O 



HCOUCCDi ilODEL (t) 




SOI IMO 



io» oil LCds 101 « 1MNI im ^uu tseo 

ost ctnnmAnoN imm tm 
mi, acsL Mood m 



191. 



EXHiBIT e-4. COMPARISON OF QSL CERTinCATION UNDER 
FULL AND REDUCED UM MODELS USING BEST DATA 
FOR UNDERGRADUATE STUDENTS 
(PERCENTAGE BLOCK CHART) 
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EXHIBIT e-5. COMPARISON OF GSL CERTIF1CATIGN UNDER 
FULL AND REDUCED UM MODELS USING REPORTED DATA 
FOR GRADUATE STUDENTS 
(PERCENTAGE BLOCK CHART) < Q - 
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EXHtBtT COMPARtSOH OF GSL CERTinCATION UNDEh 
FULL AND REDUCED UM MODELS USING BEST DATA 
FOR GRADUATE STUDENTS 
(PERCENTAGE BLOCK CHART) j ^ G 



with increased certification, 64 percent increased to the full amount 
($5,000) from a certification of $2,501 to $4/999, 

6.4.3 Student Certification Error Under the GSL Models 

Table 6-11 summarizes the effects of student misreporting on GSL 
certification under the full and reduced formulae. The overcertif ication 
rate under the reduced formula is less than half that under the full 
formula. Overcertif ications total $1,302.6 million under the full 
formula and $506.8 under the reduced formula/ a difference of $795.8 
million. These Tigures indicate that GSL certification is less sensitive 
to student misreporting under the ;educed formula than under the full 
formula. 

6.4.4 Conclusions of GSL Alternatives 

Like the effects of the reduced formula in the Campus-Based programs, 
the effects in the GSL program suggest that reducing the UM formula has 
promise in increasing the accuracy of determining certifications in the 
GSL program. The distribution of certifications under the reduced 
formula ^approximated the intended distribution very closely. In 
addition, the effects on certification for both graduate and 
undergraduate students is such that most students receive nearly the same 
certification under the reduced formula. Finally, student error under 
the reduced formula is significantly lower than under the full formula. 



6-33 



Ta5L£ 6-11 



GSL iTODENT CERTIFICA'^IOH ERROR* UNDER THE FULL AND REDUCED 

FORMULAE, 1985-86 



NO ERROR 

(Within $50) 
Percent 



OVERCERTIFICATIONS 



Percent 



Total ** 
Mean ($ Millions) 



Full Formula 
Reduced Formula 



69. 1 
85.7 



30,9 
14-3 



1,204 
1,014 



1,302.6 
506.8 



* All recipients go through need analysis, and no prospective income used. 

** Due to a revision in the estimate of total GSL loan volume, these figures 
snould be reduced by approximately 10 percent. 



6.5 CONCLUSIONS 



Shortening the UM formula shows promise in increasing the accuracy 
and efficiency of determining need in Campus-Based programs^. The 
distribution of need under a reduced formula overcomes, to a degree, the 
distortions from the intended distribution caused by student reporting 
error. The distribution of need under the reduced formula appears 
especially attractive since th^ target group of recipients, those in the 
lowest inf'ome groups for independent and dependent recipients, gain in 
need relative to the other income groups. 

Like the effects of tLe reduced formula in the Campus-Baseu programs, 
the effects in the GSL program suggest that reducing the UM formula has 
promise in increasing the accuracy of determining certifications. The 
distributon of certifications under the reduced formula approximated the 
intended distribution very closely. In addition, the effects on 
certification for both graduate and undergraduate students is such that 
most students receive nearly the same certification under the reduced 
formula. Finally. GSL student error under the reduced formula is 
significantly lower than under the full formula. 



^ Similar analysis of data element reduction for the Pell Grant 
program completed i Stage One also showed encouraging results. 
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Simplification imolves many aspects other than just the formulae. 
Sifuplification also includes the structure of the delivery system as veil 
as changing the focus of quality assurance activities from 
process-oriented activities to results-oriented requirements. These 
activities focus on aspects of the delivery system that cause problems 
Title IV-wide. A separate volume for this study. Delivery System Quality 
Improvements , explores these aspects more deeply. 

In summary, corrective actions taken to date have achieved some 
success in reducing error, but high levels of residual error remain. 
This volume has presented analyses of some current and potential 
cc^'rtjctive actions aimed at first, making continued improvements in 
quality through expedient, short-term activities and finally, making 
major improvements in quality through changes in procedures, data, and 
levels of authority and accountability. It is hoped that ED and other 
stake holders will join the challenge in improving the quality and equity 
of student financial aid delivery. 
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DESIVATIOH OF ROOT MEAN SQUARSD DIFFBRENCE 

In comparing pairs of models, it is convenient to develop a measure 
of distance between models. The Root Mean Squared Difference is an 
adjusted measure of distance useful in comparing pairs of models. The 
derivation of this measure is described below. 



The Sum of Squared Differences (SSD) is calculated by summing the 

squareo difference in need (or certification) bocween two models over all 

recipients. The Mean Squared Diffv>rence (MSD) is the quotient of the SSD 

and the number of recipients. Root Mean Squared Difference (RMSD) is the 
square root of the MSD, 



SSD = i=l{Xii - X2i)^ 

where X^^ is the need under the first model for recipient i 
X2i is the need under the second model for recipient i 
n is the number of recipients 

MSD = SSD/n 

RMSD = -/MSD 

An alternative measure of closeness is the Mean Absolute Difference 
(MAD), 



MAD = (1/n) 



i=l l^li - ^2il 

The Root Mean Squared Difference is a more desirable measure because 
larger differences are weighted more heavily than smaller differences. 
Since the detection of large changes is important, the RMSD is the better 



measure. 



APPENDIX B 



IerJc 



203 



The tables in this appendix provide information on how prospective 
and base year models affect recipients in the Pell and Caxnpus-Based 
programs* The tables presented are cL'oss-»;a?>ulations of awards (Pell 
program) or need (Campus-Based programs), by amount of award or need, 
using best and reported data in the prospective and base year income 
models. The tables supplement the e::hibits in Chapter 3* 
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TABLE B-1 

U|I*«6 WOKTCD DATA 
TABLE 01^ PflNOLKC HY ^i4N0.fl»< 
^HMO«HO RE^O^tTCO PnGS^CCTlVE PUKD.RN Rf^nRteO flASE.YCAfl 
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200 0» L 

ess 
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90 



10 
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«.?0 1 
73, T7 1 
as.«2 1 


1,09 • 

13. 0« • 


S 1 

T 1 

0.91 1 

a.2o t 

• .39 1 


9 1 

0*36 t 
3,20 1 
1.35 1 


J 1 

0.55 1 
9.92 1 
2.13 1 


O 1 

1 1 1 1 1 


2U1 • 1000 1 


10 t 
2.«1 1 
3a, oa t 
16.33 i 


17 1 
3^10 • 
3*^17 1 
3«.9A 1 


5 1 
0,91 t 
10. Aa 1 

a.l9 1 


S 1 
0*91 1 

10,6a 1 
3.31 • 


a 1 
0.73 1 
0.51 1 

2.0a 1 


a7 
0.56 


tool • 2^00 1 


2u 1 
1.*a 1 
1«.3S 1 
20. ai i 


12 1 
2,19 1 
11,01 1 
2«.09 t 


97 1 
1ft. S« I 
52.29 1 
90.00 1 


ia 1 

?;55 1 

12,0a » 
9.35 • 


6 1 
1.09 t 
5.50 1 
a. 26 t 


1 00 
19>^ 


2501 • SOOO ^ 


13 t 
2,57 1 
7.74 1 
13.27 1 


9 1 

1^65 • 
5,36 1 
19.57 1 


35 1 
6,10 1 
20.03 1 
30,70 1 


102 1 
10,50 1 
60^71 1 
60.0ft 1 


9 1 
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The following paper is reprinted with the permission of the National 
Association of Student Financial Aid Administrators (NASFAA). The paper 
has been drafccd as a working document and is intended to generate 
discussion on the subject of need analysis reform. Hence^ the paper does 
not represent the final position of NASFAA on the topic of need analysis 
reform, and should not be interpreted as such. 



NEED ANALYSIS: THOUGHTS FOR REFORM 
A Report of the NASFAA N».ed Analysis Standards Committee 



INTRODUCTION 

The NASFAA Need Analysis Standards Committee (NASC) is representative 
of all sectors of NASFAA membership. The Committee s responsibility to 
membership is to review, discuss and evaluate all aspects of need analysis 
with regard to financial aid administration, and to make recommendations to 
National Council on the basis of such review and evaluation. 

The 1984-85 Need Analysis Standards Committee began discussions which 
suggested the conceot of need analysis should be examined as a *^o^^' 
rather than reviewed for annual updates. Committee membership for 1985-8b 
remained the same in order to facilitate continued discussion and 
development of specific recommendations to National Council with regard to 
a review of need analysis standards as they exist currently. 

This report is a result of the discussions of those two years. It is 
intended to provide a basis for proactive input which will result ir a 
strategy to effectively distribute financial aid funds from all sources to 
all applicants. It is a document driven by the need to form consensus 
around complex issues and should be received in that context. 



BACKGROUND 

Need Analysis was the term originally applied narrowly in the financial 
aid community to the process of evaluating financial need for funds 
controlled by the institution. Need analysis now broadly applies also to 
processes used for program eligibility determination beyond direct control 
of the institutional aid administrator, such as eligibility for Pell Grants 
or Guaranteed Student Loans. The expansion of the definition to address 
program eligibility issues has confounded discussion of the system. 
Because more and varied publics are impacted by the need analysis system, 
there is a broad interest m its design and output. 

The system began when private agencies designed and implemented a 
method of need analysis for institutional purposes. It was later used for 
determining federal campus-based awards. This *as followed by the federal 
government designing and implementing another method and procedures for 
Pell Grant (then BEOG) application and eligibility purposes. The 
Guaranteed Student Loan (GSL) program was subject to yet another system for 
determining need. 

In an effort to reduce the duplicative efforts of Pell processin- and 
the standard application for federal campus-based programs, a common set of 
data elements was developed. Questions needed for both Pell Grant 
determination and the financial need evaluation for campus based aid are 
now on one form, the common form for "multiple data entry" (MDE). 
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. t ifHnio entry was intended to better serve all 
The development of multiple jiy, processors agreed on 

users of the need analysis system. Simultaneously, p^o ^^j^^ ^^^^^ 

the concept of ""^t°:irrhfr?rS;;s retain;d 'for'Itud^nts applying for 
on adjusted gross income charts, was retameu 

GSL. 

The use of one form reduced paperwork for tHj;«jo;ity °f aid filers,^ 

^rnr5e^srg^^L%rir:^f.:pt^ngTo%:^^^^^ 

parties. 

=rii!Uy Jr^eSral L not con,.der,d such b.s.c Issues as: 
!s the current need analysis syste,n the most effective? 

. What components are necessary for determining a family's 
ability to pay for educational costs? 

What does a reasonable method of need analysis require and generate? 
Consideration of these questions d"" J'"™ ^"UranHrsIs"" 

need analysis system that needs a thorough review. 

The Meed Analysis Standards Committee has attempted to review the 
current ^yfteS^lnS to propose more ^ T^ZZtlo^^"." 

rrr::t';;rr;r"trt"hr?et3i:^lhl ^-eU^L -nd concerns debated by the 
coLittee and offers a committee vie. as a result of the debate. 
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The oresentation of t^^ese considerations for need analysis reform is 
not meant to be all-inclusive. It offers no specific revised formulas, 
taxation rates, or mathematical calculations. It is a reaction to 
questions raised when one moves away from the detail and examines the 
following points: 1.) public perception, 2.) consumer response. 3, ) wnat 
IS realistic, and 4,) what is essential. 



SIMPLICITY 

The current application system suggests simplicity because one form is 
utilized to serve several purposes. However, completion of the single ^or^n 
IS not simple. F^amil-^es must work their way through detailed instructions, 
numbered and color-coded sections, code lists, tax forms, and income and 
asset valuations, and then agree, through required signatures and release 
statements, to verify all information they h^ve provided. Aid administra- 
tors receive an analysis of the information and review it. The "^eview 
often reveals questionable information which requires further contact with 
the family and possible corrections. The review and correction cycles 
continue until information received agrees with tax returns or other 
documentation and appears valid within parameters not necessarily 
established by the a^d office. 

"hus. It appears need analysis ^s not a simple process from any 
perspective. Families, aid administrators, Drocessors, the federal 
government,- and related education organizations are all entities mvo^/e: 
in the system. Consensus building through a committee of all parties 
concerned has guided design of the curi^ent system in application and 
process and the annual changes. 

"he issue of simplicity must be brought to the forefront m developing 
a system to accomplish the purpose of need analysis. Would a system 
simpler for all users to understand result m greater accuracy? One 
✓ lewDomt suggests requesting more information tnan is now required m tne 
pursuit of accuracy. Another viewpoint, m t^^e same pursuit, suggests a 
move to collect less detailed information, ^hese are not mjtually 
exclusive views. They are bot^ important to consideration of change 
need analysis. It should also be acknowledged that complexity does not 
necessarily ensure equity, nor is fairness guaranteed by endless, 
complicated revision, "^he aid community should be movi-^g ^n the J^^ect'C" 
of accuracy with simplicity, not accuracy or simplicity. 

In our efforts to deliver dollars ^or education, barriers which 
discourage oarticipat^cn py educationally, financially, or socially 
disadvantaged consumers should be removed. Our requests for data should be 
reasonable for all consurers. The need analysis system should protect and 
encourage access, and o^^2r a fair measure of ability to pay ^or all filers 
based only on data deter-^^ned essential to that effort. 

The NASC nas been sensitive to the varied opinions in the aid 
community, yet believes c:noromise on the issue of accuracy with simplicity 
must be reconciled as an overall objective. The current application ^orms 
require responses to mo'-e tnan 70 items related to financial matters if the 
family submits a complete application witn supporting detail. Requests for 
more information beyond t^e 79-plus are not uncommon. 
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Issues debated included: At what point do we have enough information? 
At what income level does any extra financial resource make a difference in 
the estimated or realistic ability to pay? Must all appl icant<> be directed 
through questions that do not apply to alP What is gained or lost Dy the 
degree of detail now in use? Is calculated ability to pay a search for an 
exact dollar amountc or would an estimate within ranges serve just as welP 

Is it possible for fewer questions about financial matters to provide 
an acceptable estimate of ability to pay? At what point does requested 
data become an intolerable burden to the public? What specific data 
elements are significant in affecting and measuring ability to pay? If 
income items alone will provide a reasonable estimate, is it necessary to 
do expensive analysis of assets'^ 



GENERAL OBJECTIVE: The need analysis process should make delivery of 

multiple resources as simple and expeditious as 
possible for all users of the system. 



Specific Goals: 1. The application and instructions for 

collecting necessary data elements should be 
designed to be understood at the sixth grade 
reading level. The application should take 
approximately thirty minutes for the average 
filer to complete. 



2. 



Information requested for the necessary data 
elements should be readily available to the 
family and easily verified. 



3. '^he calculation of ability to pay through treatment 
of data elements should be simple 

enough to be comfortably performed manually by 
aid administrators, guidance counselors, 
parents, students, and auditors. 

4. A simple, single application and correction process 
should provide prompt results. 

5. The system should be designed to accommodate 
new technologies with simple input and output. 

5. The application should promote accurate 

reporting of information with less complex, 
more straightforward approaches. 

7. system should be designed to serve all 

programs, populations, and users. 
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FACE VALIDITY 




It"o'ld not be surprni^g th.t families -ay have a wary attitude about the 
process ?nd its results. 

The aid administrator is the frontline defense in explaining 
i.st tutional aid policy and federal program eligibility. The campus aid 
administrator is the human element in a :eemingly mechanized process and is 
he e re expected to provide logical answers to ^^"^^^"J/^^Jjr jng he 
orocess and results. It is in that one-on-one contact with families that 
face validity is important. Families with general economic or cas f ow 
pr bl^ms are'often stunned to hear -rbal ized theories of need a y i 
which have little relation to their real problems u. immediate access to 
? in for educational expenses. The system of need J"^ f^l^. ^^/"^^ 
created this effect by itself, but the current system has created a 
situation in which provision of such detailed information can be 
interpreted by families as clear representation of why financial help is 
needed and whj they deserve assistance. This attitude covers the economic 
spectrum. 

As the iid community experiences more applications from all income and 
asset levels, it becomes increasingly difficult to offer a 
explanation of the system in all instances, ^he sophistication of the 
information intimidates those from the lower socioeconomic ^^^^^^ anJ^^J^^' 
t possible for the wiser clientele to express cifficulty ,n providing the 
expected family contribution. Some families approach t e P'-^^^VI 
attftude that the system can and should be manipulated to one s advantage 

use t yd not'believe in its accuracy in reflecting their si ua ^on 
A variety of organizations openly advertise seminars describing methods for 
maniDulatinq asset and income information to ensure maximum aid 
e? g?Sn y! The aid community does not condone that manipula ion but it 
ts a reality that impacts face val.dity. The system should not set up 
4lse expectations nor should it be so complicated as to foster belief 
that every single financial aspect his been considered and duly judged. 

The aid conmunity views processing and need determination more in the 
aggregate, whereas the individual ^^amily perspective is one s^""'^^ 
personal issue. From the family perspective, if the aid administrator is 
able to provide a good financial aid package, the system works, i. the aid 
pa kage does not recognize family expectations based on their perception of 
a realistic contribution on their part, the system, and possibly the aid 
admin! traton are under suspicion. Families ^onfusec about the compexiy. 
and wary of the validity, are then expected to comprehend the explanations 
of program eMgibility. funding restrictions, and retraining options. 

Questions debated include: Is it appropriate to continue calculating 
elementary/secondary tuition allowances in the system As ou" population 
ages, what about eldercare? What is appropriate treatment °J 
fund in need analysis when families are publicly encouraged to participate 
arJ plan their own retirement? V' A is appropriate for pension fund 
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b« " o-"!hi.ld K- naScial treatment for stepchildren and 

rt^ppare^J \lfor.ttVol' lo"current asset calculations prove necessary and 
essential in calculating current ability to pay? 

Aaain the committee understands the logic and rationale offered by aid 

l^Z'. "- « « e the exception to .ny jen.r.l fin.ncl.l scheme. 
ThrSwd ana^ys s system ™st accomodate the ,«ajorUy. recoflnire that no 
sytte"'i?^ be perfect, and accept tolerable compromises tn the overall 
picture. 

Credibility and validity must be restored in all aspects of need 
analysis emulation of ability to pay is unrealistic in the mmds o 
maJy people Fairness and equity must come from straightforward, easily 
understood approaches to gathering needed information. 

GENERAI OBJECTIVE: Need analysis should represent a realistic system 

which estimates an ability to pay seen as 
reasonable by all constituents. 

Soecific Goals- 1. The system must realistically reflect how 

Specific boa(s. ^^^.^y^^ actually pay for higher education 

costs. 

2. The system should emphasize current income 
calculations and move awcy from detailed use 
of asset data in calculation of ability to pay 
rf it can be proven that assets do nov bear a 
significant relationship to actual contributions. 

3. Estimated information, which cannot be 
verified, should not be used. 

4. The system must recognize the various family 
structures impact ability to pay. 

5. The process shr^'ild rjnforce the 
primary role or the family in financing 
postsecondary education. 

6. The system must distinguish between discretionary 
and nondiscretionary income and expenses and the 
use of those items m estimating family contnbu- 



:ion. 



7. The system should be reviewed by economists and 
financial experts to ensure it is based on a 
foundation of accept3d economic principles whic^ 
recognize current realities. 
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fhff b. d«l9n.d to el-n,(n.te th, n«d for . 
large number of adjustments. 

should be restored through use of a simple, easily 
explained system. 

HISTORICAL PRESUMPTIONS 

rplim;nt:5 unSer conditions which no longer exist. 

It .ay be wise to step -ay from the detail of the c--t^tystem^to 

decide if it is accomplishing ^^^^ P"''J° JJ* calculate an exact 
current system be P'l^erved? J^o^J^ need anal ys ^^^^^ 

dollar amount a ^ ^,^3",^^=,°;hrre a method^f^^^ determining acceptable 
of education art estimated? there a mej ^nfonnation result m a 

estimates' Does the use of both income and asset 
significantly more accurate ability to pay? 

The answers to these ^^^.^^^^^^^^^^^^^ Sai:"' 
analysis system takes ^1 J * Ji^J'iJ ?y to pay may no longer 

Some established P''i"=^.P^f ,f f^iJl nuat o^^^^ As an'example. 
have the validity required to '^^^^.""Vur? differences in family economic 
contributions from assets may -^.f^^^^j p lj^al 'difference in ability 
strength, but may not reflect a ^ ° consumer debts may not 

to pay. Similarly, the J^^jt/f^^ J^?o provide .oney for college 
"^'^"^J^ thltri^ointf ::.u?re examination. 

The committee -ognizes the impact^^^ 

scrutiny with less disdain, the disruption was merited. 

The economy, financing -cjanisms credit str-tu;es,^^^savings^plans. 
and tax issues have had 519"^^^""^ mpac^ on t e planning 

to the tax structure '"^y/''" ^f.^^lf^Jo" edu ation must function within 
options. Calculations of ty to P y o .^^^^^^ deal with 

rhrrelliHJrfl'ng fa^^ies as consumers of postsecondary education 

opportuni ties. 
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GENERAL OBJECTIVE: All aspects of the data used in need analysis 

should be subjected to evaluation for 
appropriateness to the purpose of the system. 

Specific Goals: 1, Income is assumed to be central to the calculation 

of ability to pay and should be recognized as such 
in need analysis, 

2, Savings for postsecondary education expenses should 
be encouraged by the need analysis system, 

3, An assessment procedure for the non-saver should be 
explored, 

4, Allowances for expenses and assessable income must 
stand a reasonable public test for all 

consti tuents. 



STRATEGIC PLANNING 

Change has become a recognized and accepted reality in financial aid 
administration. Much of the change, though, has not resulted from long 
range thinking. It nas been an evolution motivated by reaction to 
perceived inequities and the identification of specific problems. Such ad 
hoc changes may reflect weaknesses in a system without direction or 
structure for planning change. 

It is appropriate and timely for the aid community to initiate a plan 
that outlines and confirms a specific direction. Annual changes respond to 
immediate problems tat do not necessarily indicate progress toward a 
specific goal* The continued debate on separate issues »'ithin need 
analysis can be minimal if a long-range plan is in place. A plan implies 
stability, Cv.isistency, and implementation of common goals. A plan for 
change also provides objectives from which success can be measured. 

The aid community publicly encourages families to plan for education. 
Does the aid community know its own plan? In theory, families should know 
how much they will be expected to contribute to college costs far enough In 
advance to permit thoughtful planning. Financial aid administrators should 
be able to tfssist that family endeavor, A plan for the components in need 
analysis and subsequent analysis of data for more than a twelve month 
period of time would be a move toward stability and predictability. The 
annual debate on what items remain, what gets changed, and what gets added 
to need analysis reflect the lack of an overall plan. Is it possible to 
focus on necessary components, decide on appropriate treatment in the 
methodology, and subsequently apply annual economic updates? 

Families deserve to see consistency in treatment of information from 
one year to the next. The aid community should be able to explain the need 
methodology with more assurance than we now possess. 
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Initiation of a plan will necessarily force difficult decisions. The 
choices in the end may not result in more of the same or less of the same. 
The problems brought to light may require an entirely new solution. If the 
aid community itself has no plan, some other entity will design one. 
Strategic planning means the right questions are raised and thoughtful, 
responsive answers surface as objectives. 

In order to develop a long-ran^e plan for need analysis, the aid 
community must determine the basis on which such a plan can be completed. 
Determination of that basis requires identification of primary objectives 
of need analysis. Only by having identified the purpose can a strategic 
plan for need analysis be developed. 



GENERAL OBJECTIVE: A plan for the development, implementation, and 

enhancement of a need analysis system to 
realistically predict family contribution toward 
educational costs must be produced. 

Specific Goals: 1. The plan will be written for a three to five year 

time frame. 

2. The plan must be sensitive to demographic changes 
in the population to be served by federal, state, 
and institutional aid programs. 

3. The olan must be based on generally accepted 
standards of the aid community in assessing the 
family's ability to pay. 

4. The plan should provide flexibility for change in a 
systematic, non-disruptive manner for financial aid 
filers and aid administrators. 

5. Basic components of the plan should be available to 
all publics served by the need analysis process. 
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CONCLUSION 



The need analysis system does not operate alone in th© delivery of 
student aid, but it can become a more effective part of that delivery. 
Families want to know what will be required of them and what results they 
can expect. The aid community wants to deliver money to deserving students 
effectively. Congress and the taxpayer may be significantly more 
supportive ^f they see that p:-udence and common sense are among goals of 
aid professionals. Judicious stewardship of public and private funds 
demands that our distribution system survive the "reasonable man test. 
The issues of simplicity, face validity, fairness, and planning are 
foundations on which to build. The need analysis system should be a 
rational, separate part of the financial aid process. If it is a rational 
piece of the whole, issues related to program eligibility, rationing, 
timing, verification, and delivery of dollars can be more readily resolved 
and achieve their own validity. Each element of the financial aid system 
must attempt to become an integral part of a whole based on clearly stated 
objectives. 

These issues are not resolved in this paper, but do represent thoughts 
and questions to be raised when a review of the current rystem is done. Is 
the "snapshot" still a viable approach? Should there be a distinction 
between types of assets when calculating the ability to pay? Is a 
contribution from income different from a contribution from assets^ Is 
cashing in or borrowing against an asset to meet an expected contribution 
reasonable when economic conditions make such steps difficult? Is 
educational financing an annual event? Can long-term assurances of aid 
packages be made to encourage better family financial aid planning^ 

lo there a difference m ability to pay now and the ability to pay 
over-time? How are postsecondary education payments to be reconciled with 
current and past family debt commitments? Must the Pell Grant processor 
continue as i ? Do we really have uniform methodology if institutionally 
determined variables impact the results? How can verification be 
simplified'^ Have the intricacies of the current system gone beyond the 
generally required expertise of the average aid administrator? 

This paper is presented as a departure point for positive -and 
constructive criticism of a system in need of review. The MASFAA Need 
Analysis Standards Committee members do not present it as an unreasonab"'e 
criticism of where we are but a call for an extensive review of the concept 
of need analvsis. The Committee seeks National Council endorsement of the 
concept of reform and hopes to see NASFAA submit these goals and objectives 
to other interested parties for comment. The move for future changes 
should be led by those professionals who deal with those most impacted by 
need analysis. 
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The tables in this appendix provide information on how full and 
reduced models affect recipients in the Campus-Based and GSL programs. 
The tables presented are cross-tabulations of need (Campus-Based 
programs) or certification (GSL programs), by amount of need on 
certification, using best and reported data in the full and reduced 
models. The tables supplement the exhibits in Chapter 6. 
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